$ show sipcapture/homer
Homer is a production-grade, all-in-one HEP (Homer Encapsulation Protocol) capture and API server for SIP, RTP, RTCP, OTLP, and log data. It powers Homer 11.x data lake with DuckDB, Parquet, and Apache Arrow. The system ingests packets over UDP, TCP, TLS, HTTP, and WebSocket, decodes them using a custom zero-copy SIP parser, and stores them in a columnar DuckLake (DuckDB-backed) storage with SQLite catalog, supporting S3 and Azure backends. It provides a REST API (Echo) for search, dashboards, alerts, and configuration, an Arrow FlightSQL server for Grafana integration, and a React/TypeScript dashboard with drag-and-drop widgets, call flow visualization, and even browser-based games (chess, Doom). Architecture is modular with four main components: Ingest (transport servers), Storage (DuckLake manager with sharded writers, compactor, tiered mover), Node (FlightSQL server, catalog metadata), and Coordinator (REST API, auth, Lua correlation engine, LLM integration). Data flows from HEP/OTLP/Line Protocol receivers through decoders into DuckLake, then served via FlightSQL or REST. External dependencies include DuckDB, S3/Azure SDKs, Lua (golua), OTLP protobuf, and InfluxDB Line Protocol parser. The codebase demonstrates strong engineering practices: object pools for hot-path allocations, idempotent stop patterns, retry with backoff for catalog operations, and a Lua sandbox for scripting. However, it also carries some feature creep (games, LLM) and technical debt (dual SIP parsers, SQL injection risks via string interpolation in some services, and a few commented-out tests). Overall, it is a well-maintained, high-quality observability platform for VoIP and telemetry data.
The LLM identified 6 files worth highlighting for reuse:
Core HEP packet decoder that parses HEPv2/v3/protobuf frames, extracts SIP fields, and normalizes timestamps (NTP vs Unix epoch). Uses sync.Pool for HEP struct reuse to reduce allocation on hot ingest paths. The timestamp sanitization logic handles misdetected NTP timestamps by checking year range. This component is isolated and could be reused in any HEP-compatible capture system.
Zero-allocation SIP message parser using unsafe pointer arithmetic to avoid string copies. Extracts start line, headers (From, To, Via, Call-ID, etc.), and custom headers. Configurable via ZerocopyOpts for custom header extraction and a-leg ID detection. Demonstrates advanced Go optimization techniques and is reusable in any SIP parsing context requiring high throughput.
Generic in-process pub-sub broker with a ring buffer for recent event history, per-subscriber rate limiting, and backpressure via channel blocking. Supports nil-safe disabled state and configurable buffer sizes. This pattern is reusable for any real-time event streaming within a Go application, not just HEP data.
Native parquet file compactor that merges small files into larger ones without holding a catalog lock during the merge. Uses a two-phase approach: merge files in a temporary location, then swap in a short transaction. Avoids DuckDB's memory blow-up during VACUUM. This pattern is reusable for any DuckDB-based lakehouse needing efficient compaction.
Go-to-Lua value proxy that allows Go structs, slices, and maps to be accessed from Lua scripts via metatable-based operations. Supports indexing, method calls, and iteration. Used by the correlation engine to expose SIP data to Lua. This is a standalone library that could be extracted for any Go application needing Lua scripting with native Go types.
Password hashing utility that uses bcrypt for new passwords but also supports legacy SHA256 hashes for migration from older Homer versions. Provides HashPassword, CheckPassword, and IsLegacyHash functions. The legacy compatibility layer is cleanly abstracted and reusable in any system needing gradual password migration.