LiteFS is a FUSE-based distributed filesystem for replicating SQLite databases across a cluster of machines. It operates as a Linux daemon that intercepts SQLite's file-level operations (journal, WAL, SHM) to capture every transaction. These transactions are recorded in a custom LTX (Lite Transaction) binary format and streamed to replica nodes via an HTTP/2 h2c protocol. The system uses leader election (via Consul or a static configuration) to designate a primary node that accepts writes, while replicas synchronously apply changes and enforce read-after-write consistency via TXID cookies and a proxy layer. The architecture is fundamentally a thin FUSE layer wrapping a distributed state machine: each node runs a store that manages multiple databases, tracks guards for SQLite lock compatibility (PENDING, SHARED, etc.), and exposes an HTTP API for streaming replication, manual imports/exports, handoff, and backup integration. The proxy server sits in front of the application to route read requests to the primary or a sufficiently caught-up replica, using Fly-Replay headers for transparent redirect. Backup is supported through either a file-based mechanism or the LiteFS Cloud client. External dependencies include Consul (for leader election), FUSE (required on Linux), and the underlying SQLite library (no cgo, pure Go via the ltxx package). The codebase is written in Go, with a heavy focus on correctness under concurrency—evidenced by extensive integration tests that exercise split-brain recovery, checkpoint races, and multi-node handoff. The project is in beta and follows a roadmap-driven contribution model where the maintainer rewrites feature PRs to ensure consistency.
The ReplicaDB repository is a high-performance, open-source command-line tool for bulk data replication between heterogeneous databases. It supports a wide range of data sources, including relational databases, NoSQL databases, data virtualization platforms, file formats, cloud storage, and streaming platforms. The tool is cross-platform compatible with Windows, Linux, and macOS, and leverages parallel data transfer for optimal performance and system utilization during large-scale data migrations and synchronization tasks.