DuckDB is a high-performance, embeddable, in-process analytical SQL database engine written in C++. It provides a full SQL dialect with vectorized execution, columnar storage, advanced compression (ALP, Chimp, dictionary, FSST), adaptive indexing (ART), and a rich optimizer with rule-based and cost-based transformations. The system is designed for OLAP workloads, supporting complex queries with window functions, recursive CTEs, pivot operations, and lambda expressions. It exposes multiple client APIs: a C API (v1 and v2), C++ library, Swift bindings, and a command-line shell. Extensibility is a first-class concern via a plugin system for table/scalar/aggregate functions, custom storage backends, and data format readers (Parquet, CSV, JSON, Arrow). The repository includes extensive benchmark suites (TPC-H, TPC-DS, ClickBench, LDBC, IMDB, recursive CTE) and a comprehensive test infrastructure. While primarily an application (database engine), it also contains research-grade components in query optimization, compression algorithms, and parallel execution, giving it a hybrid character.
RocksDB is a high-performance, embeddable, persistent key-value store library written in C++. It uses an LSM-tree architecture with memtables, sorted string tables (SSTs), multi-level compaction, write-ahead logging (WAL), and advanced caching (LRU, Clock, tiered secondary caches). The codebase is exceptionally well-engineered, with extensive use of templates, platform abstractions, and custom allocators to achieve high throughput and low latency on flash storage. It is not a standalone application but a library consumed by databases (e.g., MyRocks, Cassandra), storage engines, and other infrastructure. The repository includes a comprehensive public API under include/rocksdb/, a C API, Java bindings via JNI, command-line tools (ldb, sst_dump, db_bench), and a fuzz testing suite. The project is actively maintained by Meta and the open-source community, with frequent commits, rigorous CI, and performance benchmarks.
This repository contains the complete source code for SQLite, a self-contained, high-reliability, embedded SQL database engine. The codebase implements a full SQL database system including a SQL parser (generated by the included Lemon parser generator), a bytecode virtual machine (VDBE) for executing prepared statements, a B-tree storage engine, a transaction manager with WAL and journaling, and a pluggable VFS (Virtual File System) layer for OS abstraction. The architecture is a monolithic C library designed for embedding into applications, with optional loadable extensions for full-text search (FTS3/4/5), R-tree spatial indexing, JSON processing, vector similarity search (vec1), and dozens of utility functions. Key entry points include the public C API defined in sqlite3.h, the command-line shell (shell.c), and the Tcl test harness. Data flows through the SQL parser to the query optimizer (where.c), which generates VDBE bytecode executed by the virtual machine, ultimately reading/writing pages through the pager and B-tree layers. External dependencies are minimal: only standard C library and optional ICU for Unicode support. The repository also includes extensive test suites, fuzz testing harnesses, and build systems for Unix, Windows, and WebAssembly.
This repository contains the SQLite embedded SQL database engine source code, mirrored from the actively maintained SQLite project. The core is a C library that parses SQL, compiles it into a custom bytecode virtual machine (VDBE), and executes it against a B-tree storage layer backed by a transactional pager and a pluggable OS abstraction (VFS). Primary entry points are the public C API in src/sqlite.h.in, the sqlite3 command-line shell built from src/shell.c.in, and the init functions of many loadable extensions.
LiteDB is an embedded NoSQL document database library for .NET. In production it is not a server but an in-process database engine that persists BSON documents to a single file with ACID transactions, WAL/checkpointing, MVCC snapshot isolation, B-tree and vector indexes, optional AES-256 encryption, and multi-process shared-file coordination. The public API is built around `LiteDatabase`, `LiteCollection<T>`, and `LiteRepository`, with `BsonMapper` for POCO mapping and both LINQ and SQL-like `Execute` query paths. Internally, `LiteEngine` implements page-based storage, crash recovery, compaction/rebuild, system collections, and an aggressive query optimizer that rewrites predicates, selects indexes, and supports vector similarity search. The repository also ships a CLI shell, a repro runner, stress tools, and a substantial fuzzing harness used to prove durability and recovery correctness under injected faults and power loss. The architecture is a layered monolith: a client/API layer (`LiteDatabase`, mapper, SQL parser, LINQ expression translation), a query execution layer with index selection and predicate optimization, and a storage layer with buffered page cache, WAL recovery, transaction registry, and shared-mutex coordination. Write flows go from POCO through `BsonMapper` into `BsonDocument`, into transaction snapshots and the WAL, then onto data/index pages; reads flow from index seeks/scans through BSON expression evaluation into materialized documents. The core library has no third-party runtime dependencies, relying only on the .NET base class library; supplementary tools add BenchmarkDotNet, Spectre.Console, and Gemini embedding APIs. The code is actively maintained, with a very large regression suite keyed to GitHub issues, continuous fuzzing, and compatibility tooling across file format versions.
bbolt is a production-grade, embedded key-value store for Go, forked from Bolt and inspired by LMDB. It provides ACID transactions, memory-mapped zero-copy reads, and a B+tree data model with buckets and cursors. The core library is used as the storage engine for etcd, InfluxDB, Consul, and other high-throughput systems. The repository also includes a full-featured CLI tool (`bbolt`) for database inspection, integrity checking, benchmarking, and surgical repair (e.g., page copying, freelist reconstruction). The architecture is a pure-Go library with minimal dependencies, relying on platform-specific syscalls for mmap, flock, and mlock. The codebase demonstrates strong engineering practices: clean abstractions (swappable freelist implementations via an interface), extensive test coverage including fault injection and robustness tests against filesystem failures, and a well-organized CLI command structure using Cobra. While primarily a library, the CLI makes it also a standalone application for database administration.
redb is a high-performance, ACID-compliant embedded key-value store library written in Rust. It provides a B-tree-based storage engine with copy-on-write semantics, MVCC concurrency control, and support for typed tables, multimaps, savepoints (ephemeral and persistent), compaction, and crash recovery. The library exposes a clean public API centered around `Database`, `TableDefinition`, `WriteTransaction`, `ReadTransaction`, and `ReadableTable` traits, with optional derive macros for user-defined key/value types. It is designed for single-writer, multiple-reader access and supports multi-process concurrency via byte-range file locking (experimental). The codebase is extensively tested with integration tests, fuzzing, and benchmarks comparing against LMDB, RocksDB, SQLite, and fjall. Architecturally, redb uses a copy-on-write B+tree for all tables (data, system, freed pages) with a buddy allocator for page management and a region-based file layout. The page store includes a striped LRU read cache, write buffer, and dynamic cache partitioning. Commit strategies range from non-durable (no fsync) to 1PC+C (default) and 2PC (secure). Crash recovery relies on XXH3-128 checksums and optional quick-repair via persistent allocator state. The library is `no_std`-compatible with feature-gated std-dependent parts, and includes a custom spin-lock synchronization module for embedded targets. Key entry points include the `Database::builder()` and `Database::open()` methods, `TableDefinition::new()` for schema definition, and transaction methods like `WriteTransaction::commit()` and `ReadTransaction::open_table()`. The `StorageBackend` trait allows pluggable storage (file, in-memory, custom) and is used extensively in tests for fault injection and crash simulation. External dependencies are minimal (standard library, optional chrono/uuid), with benchmark crates depending on heed, rocksdb, fjall, and rusqlite for comparison. The repository demonstrates strong engineering craftsmanship with clean abstractions, careful handling of edge cases (max keys, large values, corruption), and a well-documented design document (`docs/design.md`). The codebase is actively maintained with a stable file format and backward compatibility guarantees.
H2 is a full-featured, embeddable SQL database engine written in pure Java. It operates as a JDBC driver supporting both embedded (in-process) and client-server (TCP, PostgreSQL wire protocol, web console) modes. The codebase implements a complete SQL parser, query optimizer, transaction engine with MVCC, multiple storage engines (MVStore, legacy PageStore), and a rich set of data types, functions, and compatibility modes for databases like MySQL, PostgreSQL, Oracle, and MSSQLServer. It also includes tools for backup, recovery, CSV import/export, and a web-based administration console. The architecture is modular: the `org.h2.mvstore` package provides a standalone B-tree based transactional key-value store, `org.h2.value` defines a comprehensive type system, and `org.h2.expression` offers an extensible expression and function framework. The repository is exhaustively tested with thousands of SQL scripts and unit tests covering concurrency, crash recovery, and cross-database compatibility. It is actively maintained, with recent commits and support for Java 21 virtual threads and GraalVM native-image.
BuntDB is a low-level, in-memory, key/value store library in Go. It supports custom indexing (B-tree and R-tree), geospatial queries, JSON indexing, TTL expiration, and ACID transactions with multiple readers / single writer. Persistence is via an append-only file (AOF) with automatic background shrinking. The library is designed for high performance and embeddability, and is used by projects requiring a fast, reliable embedded database. The codebase is a single package with a well-structured API, comprehensive tests, and minimal dependencies.
RoseDB is a lightweight, fast, and reliable key-value storage engine implemented in Go, based on the Bitcask storage model. It provides a simple embedded database library with operations like Put, Get, Delete, batch writes, TTL/expiry, key watching, and merge (compaction). The architecture uses a Write-Ahead Log (WAL) for persistence and an in-memory BTree index for fast lookups. The codebase is well-structured, with clean separation of concerns: the DB layer, index abstraction, record encoding, and utility functions. It demonstrates strong engineering practices including thread safety via sync.RWMutex, object pooling for performance, and a pluggable index interface. The repository includes comprehensive tests, benchmarks, and examples. It is actively maintained with recent commits and a responsive community.
LevelDB is a high-performance, embedded key-value storage library written in C++. It implements a Log-Structured Merge-Tree (LSM) architecture, organizing data into sorted tables across multiple levels to optimize write throughput and read latency. The codebase provides a stable C++ API with a C wrapper, supporting atomic batch operations, transient snapshots, and forward/backward iteration. It is designed for single-process access with no builtin client-server support, making it ideal for embedded use cases where low-latency access and crash recovery are critical. The architecture is monolithic, consisting of core database logic in the `db/` directory, table management in `table/`, and utility functions in `util/`. Platform abstraction is handled via the `port/` directory, isolating OS-specific file and thread operations. The library relies on external dependencies for compression (Snappy, Zstd) and testing (GTest), managed via CMake and submodules. It emphasizes durability through Write-Ahead Logging (WAL) and robust recovery mechanisms that validate manifests and replay logs on startup. While the codebase demonstrates exceptional craftsmanship in concurrency control and data structures, it is receiving very limited maintenance, accepting only critical bug fixes. The repository requires C++17 and CMake 3.22, which may limit compatibility with older toolchains. The C API requires careful ABI management, and platform-specific implementations in `util/env_posix.cc` and `util/env_windows.cc` risk divergence over time.
ObjectBox Java is an open-source, embedded NoSQL database library for Java and Kotlin, targeting Android, JVM, and desktop platforms. It provides a high-performance, native-backed object persistence engine with type-safe querying, reactive data observation, object relations (ToMany/ToOne), vector search via HNSW indexes, and optional client-server or mesh peer-to-peer synchronization. The code is organized into multiple Gradle modules including a core `objectbox-java` module with JNI bindings, an annotation API (`objectbox-java-api`), Kotlin extensions, RxJava 2/3 adapters, Android-specific integrations (LiveData, Admin web UI, Paging DataSource), and a mesh sync module using Google Nearby Connections. The architecture is library-centric with a clear separation between Java-facing API, internal serialization (vendored FlatBuffers), and native runtime. Extensive unit and integration tests cover core CRUD, transactions, queries, relations, sync, and error handling. The repository is actively maintained with a version 6.0.0-beta, publishes to Maven Central, and uses multi-platform CI (GitLab, Jenkins). While the codebase includes some legacy APIs and generated FlatBuffers code, the core abstractions exhibit solid software engineering practices with reusable patterns for reactive subscriptions, property converters, and cross-platform native library loading.
nedb-promises is a lightweight Node.js library that wraps the @seald-io/nedb embedded database (a fork of nedb) with a promise-based API. It provides a Datastore class that extends EventEmitter, offering methods like find, insert, update, remove, and count that return Promises or thenable Cursor objects. The Cursor class supports fluent chaining (sort, skip, limit, project) and can be awaited directly. The library also includes MongoDB-compatible aliases (insertOne, updateMany, deleteOne, etc.) and a static `create()` method that returns a Proxy to access the underlying nedb instance's properties. The codebase is clean, well-tested, and leverages modern JavaScript features (async/await, Proxy, EventEmitter). It is a classic promise wrapper with a thoughtful event system for operation success and error broadcasting.
Sled is an embedded key-value store written in Rust that provides ACID transactions, crash safety, and high concurrency. It uses a lock-free architecture based on a lock-free pagecache, lock-free log, and lock-free B+ tree with epoch-based memory reclamation. The database supports multiple keyspaces, watch prefixes, merge operators, and a crash-safe monotonic ID generator. Currently in active development with a major rewrite underway (komora project), it is alpha software with known instability and frequent breaking changes, but demonstrates advanced concurrent data structure techniques.
Sophia is an embedded transactional key-value/row storage engine written in C, designed for high performance with ACID guarantees, MVCC (Multiversion Concurrency Control), and serializable snapshot isolation. Its architecture is inspired by LSM-trees but uses a unique node/branch/region model with O(1) worst-case set/delete and amortized O(branches) get. The system supports multiple databases, secondary indexes, custom comparators, upsert callbacks, compression (LZ4, Zstd), hot backup, online compaction/checkpointing, write-ahead logging (WAL), and crash recovery with fault injection. The codebase is modular with layers: a portable standard library (allocators, VFS, data structures, iterators, compression filters), a format/schema layer, an index layer, a transaction manager with deadlock detection, a scheduler for background jobs, and a public C API (sp_*). It includes an extensive test suite with unit, functional, crash, multithread, and hermitage isolation tests. The project is no longer actively developed, and the bundled third-party compression libraries are outdated, but the design and abstractions remain of educational and archival value.