etcd is a distributed, strongly consistent key-value store that serves as a foundational building block for distributed systems, particularly Kubernetes. In production, it runs as a Raft-based cluster (typically 3-5 nodes) providing gRPC and HTTP APIs for reading, writing, watching, and leasing key-value data. The server persists state via an MVCC engine over bbolt, with a write-ahead log (WAL) and snapshotting for durability and recovery. Key client libraries exist in Go (client/v3), along with CLI tools (etcdctl, etcdutl) and embedded server support. The architecture separates consensus (Raft), storage (MVCC/WAL/backend), API (gRPC/HTTP), and client layers, with extensive observability (Prometheus metrics, OpenTelemetry tracing). External dependencies include bbolt, the etcd-io/raft library, gRPC, protobuf, and Prometheus. While primarily an application/infrastructure project, it also includes research-grade robustness testing with model checking and fault injection, but these are testing tools, not the core purpose.
TiDB is a production-grade, open-source distributed SQL database with MySQL compatibility, horizontal scalability, strong consistency (ACID), and HTAP capabilities via TiKV (row store) and TiFlash (columnar store). The architecture separates compute (TiDB Server) from storage (TiKV/TiFlash) with PD for cluster management. Key entry points: cmd/tidb-server (SQL layer), lightning/cmd/tidb-lightning (TB-scale import), br/cmd/br (backup/restore), dumpling/cmd/dumpling (export). The codebase demonstrates exceptional craftsmanship with a modular design: the parser (pkg/parser) is a standalone MySQL-compatible SQL parser used by external projects; pkg/objstore provides a unified storage abstraction (S3/GCS/Azure/local/HDFS) with compression/encryption; pkg/dxf/framework implements a distributed execution framework with scheduler/task-executor/planner; pkg/timer provides distributed cron/interval scheduling with persistence; pkg/extension enables plugin-based authentication/session-hooks/custom-functions; pkg/util/chunk/arena/zeropool/mvmap/intset/bitmap provide high-performance columnar processing and memory management; pkg/expression uses code generation for vectorized SIMD evaluation; pkg/planner implements a Cascades-based cost-based optimizer with plan caching; pkg/ddl executes online schema changes via job-based distributed backfill with schema versioning; pkg/statistics maintains histograms/CMS/FM-sketch with auto-analyze priority queues. The repo includes comprehensive testing (unit, integration, realtikv, chaos) and modern Go practices (generics, interfaces, DI).
CockroachDB is a cloud-native distributed SQL database built on a transactional, strongly-consistent key-value store (Pebble). It provides horizontal scalability, survivability against disk/machine/rack/datacenter failures with minimal latency disruption, strongly-consistent ACID transactions, and a PostgreSQL-compatible SQL API. The architecture is a monolithic Go application with a clear separation between the SQL layer (parser, optimizer, distributed execution engine) and the KV layer (Raft-based replication, Pebble storage). Key innovations include a Cascades-style cost-based optimizer with memoization, a vectorized columnar execution engine (colexec), online declarative schema changes, distributed backup/restore with incremental support, change data capture with multiple sink formats, and multi-region/multi-tenant capabilities with geo-partitioning. The codebase demonstrates exceptional engineering craftsmanship throughout - the Raft implementation, distributed SQL planner, admission control system, and time-series storage are all genuinely reusable components that showcase non-obvious solutions to hard distributed systems problems. Active development with recent commits, comprehensive CI/CD, and regular releases.
Apache Cassandra is a distributed, wide-column NoSQL database designed for high availability, horizontal scalability, and fault tolerance. The codebase implements a peer-to-peer cluster architecture with consistent hashing for data placement and tunable replication. The system provides the Cassandra Query Language (CQL) as its primary interface, with a native binary protocol for client communication. Key subsystems include the storage engine (SSTables, memtables, commit log), distributed consensus protocols (legacy Paxos and the newer Accord transactional protocol), cluster metadata management (TCM replacing gossip), and secondary indexing (SASI and Storage-Attached Index SAI). The codebase also includes operational tooling (nodetool CLI, cqlsh shell, stress testing tool, sstable utilities) and extensive testing infrastructure (unit, distributed in-JVM, deterministic simulator, property-based, and microbenchmarks). The architecture follows a modular design with clear separation of concerns: transport layer (Netty-based NIO), query processing (CQL parser, statement execution, result set building), storage (commit log for durability, memtables for writes, SSTables for reads, compaction strategies), and distributed coordination (gossip for failure detection, streaming for bootstrap/repair, hinted handoff for availability). The newer Accord protocol introduces a transactional layer on top of the existing storage engine, enabling ACID transactions across partitions. The codebase is predominantly Java with a Python-based CQL shell (cqlsh) and extensive shell-based build and CI automation. Key entry points include the Cassandra server daemon (started via bin/cassandra), the nodetool CLI (bin/nodetool) for administration, cqlsh (bin/cqlsh) for interactive CQL, and various sstable utilities (sstableloader, sstablescrub, etc.). External dependencies include Netty for networking, Dropwizard Metrics for instrumentation, Picocli for CLI parsing, and various Java libraries for compression, encryption, and serialization. The build system uses Ant with Maven dependency resolution, and the project includes Dockerfiles for containerized builds and testing. The repository also contains formal verification artifacts (TLA+ models) and a deterministic simulation framework for testing distributed correctness, reflecting a strong emphasis on reliability.
OrientDB is a production-grade multi-model NoSQL database system supporting document, graph, object, and spatial data models with ACID transactions. It provides SQL and Gremlin query languages, full-text search via Lucene, and distributed clustering with replication, sharding, and consensus-based coordination. The architecture is modular: a core storage engine (paginated clusters, write-ahead log, multiple index types), a server layer with HTTP REST and binary protocols, client libraries (Java, JDBC, Gremlin), a distributed coordination module, and an enterprise agent for monitoring, backup, and metrics. Key entry points include the server bootstrap (OServer), SQL parser (OrientSQL.jjt), binary protocol message handlers (OBinaryRequest/OBinaryResponse), and HTTP command executors. Data flows involve client-server communication over a custom binary protocol, distributed consensus for replication and failover, and a storage engine with page cache, WAL, and pluggable compression/encryption. External dependencies include Apache Lucene for indexing, Apache TinkerPop/Gremlin for graph traversals, Dropwizard Metrics for enterprise monitoring, and multiple scripting engines (Nashorn, GraalVM). The codebase is large and well-structured with extensive use of interfaces, factories, and strategy patterns, but shows some technical debt in legacy APIs (deprecated serialization, hybrid AngularJS/Angular frontend) and multiple coexisting storage versions.
OrbitDB is a serverless, peer-to-peer distributed database library built on IPFS and libp2p. It provides a Merkle-CRDT (conflict-free replicated data type) log as its core primitive, supporting multiple database types like Events (append-only log), Documents (indexed JSON), KeyValue, and KeyValueIndexed. The architecture is modular: it separates storage backends (memory, LevelDB, IPFS blocks, LRU cache), identity management (pluggable providers), access control (static ACL or dynamic ACL stored in a keyvalue database), and encryption (pluggable). The primary entry point is `createOrbitDB()`, which returns an OrbitDB instance that can open databases of any registered type. Data flows involve appending operations to an immutable, signed log of IPLD entries, syncing with remote peers via pubsub-based replication, and handling conflict resolution using Last-Write-Wins. External dependencies include Helia (IPFS), libp2p, LevelDB, and various multiformats libraries. The library is designed for both Node.js and browser environments, with a comprehensive test suite covering unit and integration scenarios.
Kronotop is a distributed database system built on FoundationDB that provides Redis-compatible (RESP2/RESP3) interfaces for multiple data models including key-value (stash), document (bucket with BSON storage and querying), and sorted sets (zmap). It features ACID transactions, secondary indexing (including vector search via HNSW), clustering, sharding, replication, and namespace isolation for multi-tenancy. The architecture is modular with clear separation between network handling (Netty), protocol processing, service layers (transaction, session, data services), and storage (volume-based segment store). Key entry points include the server entry point (KronotopInstanceStarter), CLI (kronotop-cli), and control tool (kronotop-ctl). The system relies on external dependencies such as FoundationDB, Netty, Jackson, Lettuce, Guava, JVector, and Typesafe Config. The codebase demonstrates strong engineering practices with reusable components like versionstamp utilities, retry mechanisms, and protocol handling.
Databend is an open-source, cloud-native distributed SQL query engine and data warehouse, built in Rust. It provides a full SQL interface for analytical workloads, supporting operations such as SELECT, INSERT, COPY INTO/OUT, MERGE INTO, and complex aggregations with window functions, subqueries, and joins. The architecture follows a storage-compute separation model: a metadata service (Meta) manages schemas, users, and catalogs via a Raft-based consensus layer, while query nodes handle SQL parsing, planning, optimization (Cascades optimizer with rule-based and cost-based passes), and distributed execution over a pipelined framework. Multiple storage engines are supported, including a native columnar Fuse engine optimized for cloud object stores (S3, MinIO, Azure Blob), as well as external formats like Iceberg, Delta Lake, Hive, Paimon, Parquet, and ORC. The system exposes access via MySQL wire protocol, HTTP API, Flight SQL, and gRPC, and includes enterprise features such as data masking, row access policies, materialized views, streams, network policies, and role-based access control.
RethinkDB is an open-source, distributed, NoSQL database designed for real-time applications. It stores JSON documents and supports automatic failover, sharding, and replication. The system exposes a powerful query language (ReQL) with support for joins, aggregations, geospatial queries, and real-time change feeds. The architecture is built on a custom coroutine runtime for efficient concurrency, a log-structured storage engine with B-tree indexes, and a peer-to-peer clustering layer using Raft consensus for metadata management. External dependencies include OpenSSL, Protobuf, jemalloc, zlib, and QuickJS for JavaScript execution. The codebase demonstrates strong engineering in distributed systems, but is now stale with limited active maintenance and significant technical debt from outdated C++ practices and vendored third-party libraries.