$ show cloudamqp/lavinmq
LavinMQ is a production-grade, RabbitMQ-compatible message broker implemented in Crystal. It speaks AMQP 0-9-1, AMQPS, MQTT 3.1/3.1.1, and WebSocket transports, and exposes a RabbitMQ-style HTTP management API plus a static web UI. The broker provides durable queues backed by memory-mapped segment files with fsync-batched persistence, stream queues with consumer offsets and filtering, priority/delayed/dead-lettered/deduplicated queues, publisher confirms, transactions, policies, vhosts, users/permissions, and clustering via etcd-based leader election with ISR replication. It also implements federation and shovels for cross-broker message movement, and ships lavinmqctl for administration and lavinmqperf for benchmarking. Architecturally this is a monolithic Crystal server process that binds AMQP, MQTT, HTTP metrics/management, and control sockets. Protocol connections are handled by fiber-per-connection servers sharing vhost/queue/exchange primitives. Durable state flows through definitions stores, message stores, parameter stores, and an etcd-coordinated replicator in clustered mode. The codebase separates protocol-agnostic core components (vhost, message, queue, exchange) from protocol adapters (AMQP/MQTT), HTTP controllers, and a custom vanilla-JS frontend. The primary data flow is: producers publish via AMQP/MQTT, exchanges route to bound queues, consumers receive; persistence writes mmap segments and confirms only after fsync or in-sync replica acknowledgment. The project is actively maintained with recent commits, extensive CI including compatibility runs against Bunny, rabbit-hole, and RabbitMQ Java client, plus packaging for Docker/DEB/RPM. The code has several genuinely reusable low-level components such as CIDR/IP matchers, memory-mapped file abstractions, consistent hashing implementations, and topic matching trees. Some maintenance burden comes from custom Crystal stdlib patches and low-level Linux I/O optimizations, but the codebase tracks recent Crystal releases. Known limitations such as MQTT QoS 2 downgrade, PROXY v2 IPv6 gaps, and deprecated config aliases are documented rather than hidden.
The LLM identified 6 files worth highlighting for reuse:
Self-contained etcd v3 client implementing key-value transactions, leases, elections, watch streams, connection pooling, and endpoint failover. Demonstrates non-obvious protocol handling including raw HTTP/1.1 chunked parsing and typed error classification, and can be extracted for any Crystal service needing etcd coordination.
Memory-mapped file abstraction with lease-based concurrent close protection, msync/mremap/truncate support, and sync bookkeeping. Encapsulates low-level OS page mapping in a reusable way for any high-throughput append-only store.
Pure CIDR/exact IP matcher supporting IPv4-mapped IPv6 and efficient prefix matching. Self-contained utility suitable for access control, trusted-proxy detection, and network policy evaluation in any application.
Implements ring-based consistent hashing for stable key-to-node assignment with minimal remapping on membership changes, complemented by a jump-consistent-hash variant. Reusable for load balancers, partitioners, cache sharding, and cluster-aware routing.
Segment-based binding-key matcher and zero-allocation routing-key iterator used by AMQP topic exchanges. The wildcard matching algorithm is independent of the broker and reusable for event-routing, pub/sub filtering, or generic topic-pattern matching.
Generic subscription trie for hierarchical topic matching used for MQTT subscriptions and topic permissions. Self-contained tree structure with wildcard support, reusable in any pub/sub system requiring efficient topic matching.