Watermill is a Go library for building message-driven and event-driven applications. It is not a standalone service; applications embed it and use the core Publisher/Subscriber interfaces to communicate with brokers such as Kafka, RabbitMQ, NATS, Redis Streams, Google Cloud Pub/Sub, AWS SNS/SQS, SQL databases, or an in-memory Go channel. This repository contains the core library, reusable components, runnable examples, and small CLI tools, while broker-specific adapters and some integrations live in separate modules. The architecture is layered. At the bottom are the low-level Pub/Sub abstractions and the Message object with UUID metadata, payload, and Ack/Nack lifecycle semantics. The middle layer is the Router, which wires handlers into consumption loops and supports middleware and publisher/subscriber decorators. On top of that, optional components implement CQRS event/command buses and processors, topic forwarding, request-reply, delayed publishing, metrics, fan-in/fan-out, and requeuing. Middleware covers retry with exponential backoff, poison queues, deduplication, throttling, circuit breaking, correlation IDs, timeout enforcement, and panic recovery. The repository is extensively tested, including a universal pub/sub conformance suite intended for adapter authors to reuse, and GoChannel serves as a complete in-memory reference implementation. The examples demonstrate real production patterns: CQRS with protobuf, ordered events, exactly-once delivery counters, outbox/transactional event forwarding, server-sent event feeds, webhook ingestion, and consumer groups. There is no data research component; this is purely a software library/application framework.
This is the official NATS C client library (nats.c), a production-grade C library providing full support for the NATS messaging system. It implements core publish/subscribe, request-reply, queue groups, and advanced features including JetStream (streams, consumers, key-value store, object store), a microservices framework, and legacy NATS Streaming support. The library is designed for high performance and reliability, with built-in reconnection, TLS/SSL, authentication (NKeys, JWT), and event loop integration (libevent, libuv). It exposes a comprehensive C API with hundreds of functions, and includes extensive examples and documentation. The codebase is well-structured with platform abstractions for Unix and Windows, and internal modules for protocol parsing, connection management, threading, and memory management. While the library is actively maintained and widely used, it carries some technical debt from legacy Streaming support and a large, tightly-coupled internal architecture.
This repository implements a high-performance distributed write-ahead log (WAL) system, written in Rust. It comprises a core WAL library (walrus-rust) with configurable storage backends (mmap, fd, io_uring), a Raft consensus library (openraft) for distributed metadata management, and a distributed application (distributed-walrus) that provides a TCP-based client-server architecture for topic-based streaming with lease-based write fencing and automatic segment rollover. The system is benchmarked against Kafka and RocksDB, includes a formal TLA+ specification, and is designed for crash-safe, low-latency persistent streaming. The architecture is layered: the Walrus WAL engine provides durable, topic-partitioned log storage with configurable fsync schedules and read consistency modes. The Octopii framework layer integrates QUIC transport and a flexible state machine. Openraft (vendored) provides full Raft consensus with leader election, log replication, snapshotting, and linearizable reads. The distributed-walrus application ties these together with a node controller, metadata state machine, and monitor loop for segment rollover. While primarily an application, the repository has strong research elements: extensive performance benchmarks (scaling, batch, multithreaded), formal verification via TLA+, and a focus on characterization of trade-offs between durability, latency, and throughput. The codebase is well-structured with clear abstractions, though it contains some duplicated code across the walrus and octopii directories and a large vendored copy of openraft.
This is a Prometheus exporter for NATS server monitoring. It polls NATS server HTTP monitoring endpoints (varz, connz, subz, routez, healthz, jsz, etc.) and exposes metrics in Prometheus format. The architecture is a standalone binary with an embeddable library API. Key entry points are the CLI (main.go) and the exporter.NewExporter function. Data flow: on each scrape, collectors make HTTP requests to NATS monitoring endpoints, parse JSON responses, and convert them to Prometheus gauge metrics. The exporter supports TLS, basic auth, custom metric prefixes, and multiple NATS servers (though one per exporter is recommended). It includes pre-built Grafana dashboards for both NATS core and JetStream metrics. The codebase is well-structured with separate packages for collectors, exporter, and test utilities. It uses reflection-based metric extraction for simple endpoints and custom collectors for complex ones like JetStream, connections, and gateways. The project is actively maintained with recent commits and modern dependencies.