CockroachDB is a cloud-native distributed SQL database built on a transactional, strongly-consistent key-value store (Pebble). It provides horizontal scalability, survivability against disk/machine/rack/datacenter failures with minimal latency disruption, strongly-consistent ACID transactions, and a PostgreSQL-compatible SQL API. The architecture is a monolithic Go application with a clear separation between the SQL layer (parser, optimizer, distributed execution engine) and the KV layer (Raft-based replication, Pebble storage). Key innovations include a Cascades-style cost-based optimizer with memoization, a vectorized columnar execution engine (colexec), online declarative schema changes, distributed backup/restore with incremental support, change data capture with multiple sink formats, and multi-region/multi-tenant capabilities with geo-partitioning. The codebase demonstrates exceptional engineering craftsmanship throughout - the Raft implementation, distributed SQL planner, admission control system, and time-series storage are all genuinely reusable components that showcase non-obvious solutions to hard distributed systems problems. Active development with recent commits, comprehensive CI/CD, and regular releases.
Apache SeaTunnel is a high-performance, distributed data integration platform designed for synchronizing vast amounts of data across diverse sources and sinks. It provides a unified framework for batch and stream processing with support for Change Data Capture (CDC), schema evolution, and multi-engine execution (SeaTunnel Zeta, Flink, Spark). The platform features a plugin-based architecture with over 160 connectors for databases, message queues, cloud services, and file systems, along with a web-based UI for job monitoring, CLI tools for job submission, and SDKs for programmatic access. Enterprises like JP Morgan, Tencent Cloud, and Bytedance use it for production data integration workloads.
go-mysql is a pure-Go library for the MySQL and MariaDB ecosystem. Rather than a single application, it exposes a layered set of packages and command-line tools: a low-level packet/protocol layer, a full client connection library, a MySQL-compatible server framework, a binlog replication parser and syncer, a higher-level change-data-capture (CDC) engine called canal, a mysqldump wrapper, and a database/sql driver. The command binaries (`go-mysqlbinlog`, `go-mysqldump`, `go-canal`, `go-mysqlserver`, `go-binlogparser`) demonstrate concrete uses of the library, but the real product is the reusable code surface. The core data flow for replication is: the `replication.BinlogSyncer` connects as a MySQL replica, registers for binlog dump, receives raw binlog frames, and parses them into typed events (`RowsEvent`, `TableMapEvent`, `GTIDEvent`, `QueryEvent`, etc.). The `canal` package layers on top of this by first taking a snapshot through `dump.Dumper` and then streaming subsequent binlog events through a handler interface, making it suitable for feeding downstream stores like Elasticsearch or Redis. The `client` and `server` packages implement the query side of the protocol, including authentication, prepared statements, streaming result sets, TLS, compression, and connection pooling. The `driver` package adapts this to Go's standard `database/sql` interface. The codebase is mature and heavily tested, with CI matrices covering MySQL 8.0/8.4/9.7 and MariaDB 10.11/11.4/11.8, plus fuzz test corpora for GTID and binlog decoding. It also contains notable compatibility engineering: hand-written decoders for MySQL binary JSON, decimal, `datetime2`, `time2`, GTID sets, and row events, all validated against live-server behavior. The main risks are protocol drift and manual maintenance: the error-code mapping ends around MySQL 1863, some binary resultset helpers are deprecated or inconsistent, and much of the value depends on keeping pace with new MySQL/MariaDB binlog event formats. Overall, this is a high-quality, industrially relevant library with strong reuse potential for anyone building MySQL tooling, CDC pipelines, or protocol-compatible servers in Go.
This is Alibaba Canal, a mature, production-grade MySQL binlog change data capture (CDC) system. It simulates a MySQL slave to receive binlog dumps, parses all binlog event types (DDL, DML, GTID, etc.), and delivers the change data to downstream consumers via TCP, Kafka, RocketMQ, RabbitMQ, or Pulsar. It also includes a set of pluggable adapters (via SPI) that can synchronize changes directly to relational databases (RDB), Elasticsearch (6.x/7.x/8.x), HBase, Kudu, Phoenix, ClickHouse, Tablestore, or a simple logger. The system provides high availability through ZooKeeper-based leader election, a web-based admin console (Vue.js + Spring Boot), and Prometheus metrics for monitoring. The architecture is modular with clear separation of concerns: parsing, event sink/ordering, buffered store, metadata management, MQ connectors, and adapter plugins. While the codebase shows its age in places (Netty 3.x usage, some deprecated APIs, duplicated adapter logic across ES versions), it remains a robust and heavily used infrastructure component in many production deployments.
Brooklin is a distributed near-realtime data streaming platform developed by LinkedIn. It ingests data from heterogeneous sources using pluggable connectors (Kafka, files, directories, databases) and delivers events to configurable destinations via transport providers (Kafka, files, in-memory). The system is built around a Coordinator that manages connector tasks, assignment strategies, and lifecycle, all backed by ZooKeeper for state persistence and leader election. A REST.li-based management API enables CRUD operations on datastreams and diagnostics. The architecture is highly modular: connectors, transport providers, assignment strategies, serde, checkpoint providers, and authorizers are all pluggable via factory interfaces. Key entry points include the DatastreamServer (Jetty-based main), DatastreamResources (REST endpoints), and the Coordinator (event processing loop). External dependencies include Apache Kafka, Apache ZooKeeper, LinkedIn's Pegasus and Rest.li frameworks, Jetty, Netty, and ParSeq for asynchronous operations. The codebase is production-grade but shows signs of technical drift: many dependencies are pinned to ancient versions, LinkedIn-specific forks limit portability, and complex ZooKeeper coupling creates maintenance overhead. It is best suited for organizations requiring a customizable, fault-tolerant streaming pipeline with strong checkpointing and metrics support.