$ show ThreeDotsLabs/watermill
Watermill is a Go library for building message-driven and event-driven applications. It is not a standalone service; applications embed it and use the core Publisher/Subscriber interfaces to communicate with brokers such as Kafka, RabbitMQ, NATS, Redis Streams, Google Cloud Pub/Sub, AWS SNS/SQS, SQL databases, or an in-memory Go channel. This repository contains the core library, reusable components, runnable examples, and small CLI tools, while broker-specific adapters and some integrations live in separate modules. The architecture is layered. At the bottom are the low-level Pub/Sub abstractions and the Message object with UUID metadata, payload, and Ack/Nack lifecycle semantics. The middle layer is the Router, which wires handlers into consumption loops and supports middleware and publisher/subscriber decorators. On top of that, optional components implement CQRS event/command buses and processors, topic forwarding, request-reply, delayed publishing, metrics, fan-in/fan-out, and requeuing. Middleware covers retry with exponential backoff, poison queues, deduplication, throttling, circuit breaking, correlation IDs, timeout enforcement, and panic recovery. The repository is extensively tested, including a universal pub/sub conformance suite intended for adapter authors to reuse, and GoChannel serves as a complete in-memory reference implementation. The examples demonstrate real production patterns: CQRS with protobuf, ordered events, exactly-once delivery counters, outbox/transactional event forwarding, server-sent event feeds, webhook ingestion, and consumer groups. There is no data research component; this is purely a software library/application framework.
The LLM identified 5 files worth highlighting for reuse:
Implements configurable exponential backoff retry middleware with jitter, max retries, configurable intervals, ShouldRetry predicates, retry hooks, and optional context reset. It wraps a standard message.HandlerFunc, is self-contained, and can be reused in any Watermill router or standalone message pipeline.
Routes messages that exhaust retries to a separate dead-letter topic while preserving the original message and attaching metadata about the failing handler, subscriber, topic, and error. This is a practical, reusable dead-letter-queue pattern that works with any Publisher/Subscriber pair.
A comprehensive conformance suite for validating any Pub/Sub implementation against Watermill semantics: ack/nack behavior, ordering guarantees, persistence, stress test parallelism, subscriber multiplication, and feature flags. Adapter authors can import or model this harness to prove correctness, making it a rare and highly reusable testing asset.
Bridges messages between arbitrary Publisher/Subscriber pairs, optionally wrapping messages in an envelope to preserve routing metadata and with configurable acknowledgment behavior. It is useful for outbox relay, topic replication, or broker migration, and is decoupled from specific broker implementations.
Allows multiple CQRS event handlers to share a single subscription and dispatches each event to the handlers that are interested, reducing consumer count while preserving per-handler error isolation. Demonstrates an advanced group-consumer pattern that is reusable in event-driven services.