RisingWave is a production-grade distributed streaming SQL database. The codebase implements a multi-node cluster composed of SQL frontend nodes (parser, binder, optimizer, planner), compute nodes running both streaming executors and batch execution, a meta service for catalog/scheduling/checkpoint coordination, and a custom LSM-tree storage engine (Hummock) over object storage. It speaks the PostgreSQL wire protocol, supports incremental CREATE MATERIALIZED VIEW over streaming inputs, and serves real-time queries and sinks. The repository is primarily an application, despite containing extensive benchmarks and deterministic simulation testing: those artifacts validate distributed-system correctness rather than performing data research.
Apache Iceberg is a high-performance table format for analytic data lakes, providing ACID transactions, snapshot isolation, schema evolution, and time travel. The repository contains the Java reference implementation, including core libraries for table metadata management, file format support (Parquet, ORC, Avro, Arrow), expression evaluation, encryption, and a REST catalog protocol. It integrates deeply with Spark (versions 3.5, 4.0, 4.1, 4.2), Flink (1.20, 2.2, 2.3), Kafka Connect, Hive, and cloud object stores (AWS S3, Azure ADLS, Aliyun OSS, GCS). The architecture is modular: the `api` module defines public interfaces (Table, Catalog, Scan, FileIO), `core` provides implementations and utilities, and separate modules handle engine-specific integrations and cloud backends. Key entry points include the Table interface for DDL/DML operations, Scan for query planning, FileIO for storage access, and the REST catalog client for remote metadata management. Data flows involve reading/writing manifest files, applying expressions for predicate pushdown, and coordinating commits via optimistic concurrency control. External dependencies include Apache Parquet, ORC, Avro, Arrow, and various cloud SDKs, but the core is designed to be lightweight and extensible.
Jikkou is a declarative infrastructure-as-code tool for managing Apache Kafka and related data infrastructure (Schema Registry, Kafka Connect, Apache Iceberg, Confluent Cloud, Aiven, AWS Glue). It provides both a CLI and a REST API server that reconcile desired resource state (defined as YAML/JSON files) against live cluster state using a Kubernetes-style resource model (apiVersion, kind, metadata, spec). The system computes diffs between declared and actual configurations, then applies only the necessary changes via pluggable controllers and change handlers. Architecturally, the project is a multi-module Maven project with a core library that defines extension points (collectors, controllers, validations, transformations, health indicators, change reporters) and provider modules that implement these interfaces for specific platforms. The CLI uses picocli for command parsing, and the REST API server is built on Micronaut with HATEOAS links. Both share the same core API and extension registry. A GraalVM native-image build is supported for the CLI. The codebase demonstrates strong software engineering with clean interfaces, builder patterns, annotation-driven extension metadata, and comprehensive unit and integration tests using Testcontainers. Some code duplication exists across provider modules, and the large number of generated model classes introduces maintenance overhead. Overall, it is a well-crafted, production-ready tool for Kafka ecosystem resource management.