M3 is a production-grade, horizontally scalable distributed time series database and metrics platform, originally built at Uber. It provides a storage engine (M3DB) with custom M3TSZ compression, a PromQL-compatible query engine (M3Query), a metrics aggregation service (M3Aggregator), and a coordinator for ingestion and routing. The system ingests metrics via Prometheus remote write, Graphite (carbon), and custom Thrift/Protobuf protocols. It supports clustering, replication, sharding, and downsampling. The architecture is a set of microservices (m3dbnode, m3coordinator, m3aggregator, m3query) that communicate via gRPC, TChannel, and etcd for coordination. Key entry points are the service binaries in src/cmd/services/. External dependencies include etcd, Prometheus client libraries, and custom forks of several Go packages. The codebase is heavily optimized for performance with extensive object pooling, code generation, and custom encoding.
carbon-clickhouse is a production-grade Graphite metrics receiver and storage gateway written in Go. It ingests time-series data over multiple protocols (plain text TCP, UDP, Pickle, gRPC, Prometheus Remote Write, Telegraf HTTP JSON) and persists it to ClickHouse using the efficient RowBinary format. The system is architecturally split into receivers, a writer that rotates and compresses data files, and uploaders that batch-insert data into ClickHouse tables. Receivers handle protocol-specific parsing and drop rules; the writer manages chunking, interval-based rotation, and optional LZ4 compression; uploaders support points, index (hierarchical metric trees), tagged metrics (sorting, dedup, and URL encoding), and points-reverse storage. External dependencies include ClickHouse (HTTP interface), Prometheus protobuf definitions, gRPC, and common Go libraries. The codebase is well-tested, includes end-to-end integration tests with actual ClickHouse containers, and is actively maintained with CI/CD pipelines producing DEB/RPM packages.
Backstop is a Ruby Sinatra web application that acts as a proxy for submitting metrics to Graphite/Carbon. It accepts JSON data via HTTP POST from various sources (Collectd, GitHub webhooks, PagerDuty, and custom endpoints) and forwards the metrics to one or more Carbon/Graphite listeners. The application is structured with separate components for configuration, publishing metrics, handling authentication, and parsing Collectd data using a plugin system. The Publisher class manages a pool of TCP connections to Carbon servers, randomly selecting a connection for each metric to achieve basic load balancing. It handles reconnection by resetting the connection pool on socket errors. The Collectd parser uses a dynamic plugin system to handle different Collectd plugin types, making it easy to extend for new plugin formats.