$ show webhookx-io/webhookx
WebhookX is an open-source webhook gateway and event delivery system written in Go. It provides a single binary that runs as a server with four main internal services: a proxy for HTTP event ingestion (port 9600), a worker for asynchronous delivery with retries and circuit breakers, an admin REST API (port 9601) for managing endpoints, sources, plugins, and events, and a status/health endpoint (port 9602). Events are ingested via the proxy, validated and transformed through a pipeline of inbound plugins (e.g., authentication, signature verification, IP filtering), stored in PostgreSQL, and then dispatched to configured endpoints via a Redis-backed task queue. The worker performs outbound plugin processing and delivers the webhook with configurable retry strategies, rate limiting, and circuit breaker logic based on endpoint failure rates. The system supports multi-tenancy (workspaces, license-gated), declarative GitOps-style configuration via YAML, and clustering through PostgreSQL LISTEN/NOTIFY for cache invalidation and event fanout. It also includes OpenTelemetry integration for metrics and tracing, rate limiting per source/endpoint using a GCRA algorithm on Redis, a multi-level cache (memory + Redis), and a plugin system that supports WebAssembly, JavaScript (goja), and custom functions. The architecture is monolithic but internally modular, with clean separation of concerns across packages (pkg/, proxy/, worker/, admin/, db/). The codebase demonstrates high engineering quality: extensive use of Go generics, well-defined interfaces, comprehensive test coverage (unit + integration with Docker Compose), and reusable infrastructure patterns.
The LLM identified 5 files worth highlighting for reuse:
A generic, self-contained batch queue that collects items and flushes them when a size or time threshold is reached. Uses Go generics, provides a clean interface (Add, Flush, Stop), and includes thread-safe batching with a configurable flush interval. Could be extracted and reused in any Go project needing batched writes (e.g., to batch database inserts or log entries) with minimal modification.
A Redis-backed task queue implementing a distributed scheduling abstraction with visibility timeout for reliable processing. Uses Lua scripts for atomic enqueue/dequeue/requeue operations, supports delayed tasks via ZSETs, and includes a heartbeat mechanism. Well-isolated (only depends on go-redis and standard library), demonstrating solid async job queue design.
Implements the GCRA (Generic Cell Rate Algorithm) rate limiter backed by Redis, providing burst-aware throttling with constant memory usage. The implementation uses atomic Lua scripts for rate check and increment, supports configurable per-second/per-minute limits and burst sizes. A clean, extractable rate limiter that can be reused in any service requiring distributed rate limiting.
A time-windowed failure aggregation engine for circuit breaker state. Maintains sliding windows of success/failure counts using a series of time buckets, with pruning of expired data. Implements threshold-based failure rate calculation that automatically resets after cooldown. The aggregation logic is self-contained, testable, and could be ported to other languages or used in different failure detection scenarios.
Parses and resolves secret references in configuration strings using a `{secret://provider/path.jsonpath}` syntax. Provides a `Parser` interface and a `DefaultParser` that supports AWS Secrets Manager and HashiCorp Vault providers. The design abstracts secret resolution cleanly, allowing new providers to be added without changing configuration loading logic. A reusable pattern for securing sensitive configuration in deployment environments.