$ show hyperdxio/hyperdx
HyperDX / ClickStack is a full-stack observability platform for ingesting, storing, querying, and visualizing logs, traces, metrics, and session replays. The system is architected as a monorepo with a Next.js frontend (React, Mantine, CodeMirror, Recharts), an Express.js API backend (MongoDB metadata, ClickHouse primary data store), a CLI tool (ink-based TUI), and an OpenTelemetry collector for ingestion. Key capabilities include real-time dashboards with drag-and-drop tile layout, multi-channel alerting (webhook, Slack, incident.io), advanced search with Lucene and SQL, log pattern mining (Drain algorithm), session replay, service maps, Kubernetes monitoring, and a Model Context Protocol (MCP) server that exposes observability tools to AI assistants. The platform supports authentication via local accounts or OIDC, OpAMP agent configuration, and includes an evaluation framework (hdx-eval) for benchmarking agent performance on SRE scenarios. Although the repo is primarily an application, it contains data-intensive components like the log miner and analytics query engines that cross over into data engineering territory.
The LLM identified 5 files worth highlighting for reuse:
A self-contained implementation of the Drain log pattern mining algorithm. Includes an LRU cache for node eviction, configurable masking of common variable patterns (IDs, numbers, IPs), and a template miner that clusters logs by token similarity. The module is fully parameterized via a config object and can be extracted into any project needing unsupervised log template extraction. The code is well-structured with clear separation of concerns (config, drain core, log-cluster, lru-cache, masking, mine-patterns) and includes unit tests.
A generic MCP tool registration function that wraps tool handlers with automatic OpenTelemetry tracing (recordDuration, recordOperationOutcome) and consistent error formatting. It accepts a tool definition (name, description, schema) and a handler function, returning a registered tool object. This pattern enforces observability and error consistency across all MCP tools (alerts, dashboards, queries, sources, traces) without duplicating instrumentation logic. The function is reusable in any MCP server implementation.
A palette color picker component that restricts selection to a predefined set of color tokens (categorical palette and semantic colors) instead of arbitrary hex values. It includes a legacy token migration function (healLegacyDashboardTileColors) that converts old 'chart-1..chart-10' tokens to the new palette. The component is self-contained, uses Mantine's ColorSwatch, and exports as a controlled input. This is a reusable UI primitive for any application that requires consistent chart color selection.
A best-effort secrets redaction function designed to sanitize observability data before sending it to LLMs. It applies a curated set of regex patterns to redact PEM keys, JWTs, AWS credentials, GitHub tokens, Slack webhooks, OpenAI/Anthropic/Google Gemini API keys, and other common secret formats. The function is ordered to avoid double-redaction, includes a list of pattern names for testing, and is explicitly conservative (low false positives) with documented known gaps. It is a pure, dependency-light utility suitable for any project that needs to redact secrets from user-generated text or logs.
A lightweight sparkline component built with Recharts that supports line, area, and bar chart types. It accepts a data array, width/height, color, and a minimum value option. The component is fully self-contained with no internal dependencies beyond Recharts and React, making it easily extractable for use in tables, side panels, or any context where a compact inline chart is needed. Includes unit tests for rendering and type coverage.