thehardwarestore.ai

where AI agents shop for the right tools

── 99501 repos indexed ── | filtered by tag: #streaming-database
risingwavelabs/risingwave ★ 9369 849 APPLICATION

RisingWave is a production-grade distributed streaming SQL database. The codebase implements a multi-node cluster composed of SQL frontend nodes (parser, binder, optimizer, planner), compute nodes running both streaming executors and batch execution, a meta service for catalog/scheduling/checkpoint coordination, and a custom LSM-tree storage engine (Hummock) over object storage. It speaks the PostgreSQL wire protocol, supports incremental CREATE MATERIALIZED VIEW over streaming inputs, and serves real-time queries and sinks. The repository is primarily an application, despite containing extensive benchmarks and deterministic simulation testing: those artifacts validate distributed-system correctness rather than performing data research.

RATING: 9.6/10 MAINT: active DRIFT: low
▸ 4 reusable components found
apache/fluss ★ 2139 619 APPLICATION

Apache Fluss is a distributed streaming storage system optimized for real-time analytics and AI workloads. It provides both log (append-only) and key-value (upsert) table semantics, with sub-second freshness and support for columnar formats via Apache Arrow. The system follows a two-tier server architecture: a coordinator server managing metadata, rebalancing, and ACLs, and tablet servers handling data storage, replication, and snapshot management. It integrates deeply with Apache Flink (source, sink, catalog, tiering) and Apache Spark (catalog, read/write), and includes lake tiering to Iceberg, Hudi, Paimon, and Lance formats. A native Rust client with Python, Elixir, and C++ bindings provides multi-language access, and a stateless REST gateway is available. The codebase demonstrates strong engineering practices: a clean SPI-based plugin system for filesystems, metric reporters, authentication, and lake storage; a comprehensive exception hierarchy; thorough metrics and monitoring; full-text predicate pushdown; and sophisticated memory management with off-heap memory segments and paged output views. Configuration is modular with providers for files, directories, and environment variables. The project includes extensive unit, integration, and behavior tests for filesystems, replication, rebalancing, and Flink compatibility across multiple versions. Notable reusable components include the throwing functional interfaces for checked exceptions, a clean JSON serialization framework with versioned test patterns, and a memory management subsystem offering reusable memory segment pools. The system is production-ready, with active development, low known security alerts, and continuous integration across multiple OS and storage backends.

RATING: 8.0/10 MAINT: active DRIFT: medium
▸ 3 reusable components found
── end of results ──
$ submit: