M3 is a distributed time-series database and metrics platform originally developed by Uber and now under the Linux Foundation. It provides a full-stack solution for collecting, aggregating, storing, and querying high-volume metric data at scale. The system comprises multiple services: m3dbnode (storage node with custom encoding and persistence), m3coordinator (routing and downsampling gateway), m3aggregator (clustered metrics aggregation with quantile estimation), and m3query (PromQL/Graphite query engine). These services communicate via Thrift, gRPC, and HTTP, and are coordinated through an etcd-based cluster management layer that handles placement, leader election, and configuration distribution. The architecture is highly performance-oriented, featuring extensive object pooling, zero-copy I/O, custom bit-level encoding (m3tsz), and an inverted index (m3ninx) backed by FST segments and Roaring Bitmaps for fast metadata search. The codebase is written in a custom Go-like transpiled language and includes generated code for mocks, protobuf, and Thrift bindings. It integrates deeply with Prometheus (remote write, scrape, and query) and Graphite protocols, and provides a React-based admin UI (m3ctl) for namespace and rule management. The repository also contains a comprehensive suite of unit, integration, and property-based tests, along with Docker-based development and CI environments.
carbon-clickhouse is a production-grade Graphite metrics receiver and storage gateway written in Go. It ingests time-series data over multiple protocols (plain text TCP, UDP, Pickle, gRPC, Prometheus Remote Write, Telegraf HTTP JSON) and persists it to ClickHouse using the efficient RowBinary format. The system is architecturally split into receivers, a writer that rotates and compresses data files, and uploaders that batch-insert data into ClickHouse tables. Receivers handle protocol-specific parsing and drop rules; the writer manages chunking, interval-based rotation, and optional LZ4 compression; uploaders support points, index (hierarchical metric trees), tagged metrics (sorting, dedup, and URL encoding), and points-reverse storage. External dependencies include ClickHouse (HTTP interface), Prometheus protobuf definitions, gRPC, and common Go libraries. The codebase is well-tested, includes end-to-end integration tests with actual ClickHouse containers, and is actively maintained with CI/CD pipelines producing DEB/RPM packages.
Backstop is a Ruby Sinatra web application that acts as a proxy for submitting metrics to Graphite/Carbon. It accepts JSON data via HTTP POST from various sources (Collectd, GitHub webhooks, PagerDuty, and custom endpoints) and forwards the metrics to one or more Carbon/Graphite listeners. The application is structured with separate components for configuration, publishing metrics, handling authentication, and parsing Collectd data using a plugin system. The Publisher class manages a pool of TCP connections to Carbon servers, randomly selecting a connection for each metric to achieve basic load balancing. It handles reconnection by resetting the connection pool on socket errors. The Collectd parser uses a dynamic plugin system to handle different Collectd plugin types, making it easy to extend for new plugin formats.