$ show Mooncake-Labs/pg_mooncake
pg_mooncake is a Postgres extension that adds a custom table access method ('mooncake') backed by Apache Iceberg, enabling real-time analytics on columnar data stored in Iceberg tables. It integrates DuckDB (via pg_duckdb) as the query engine for fast analytics and provides a background worker (moonlink) that streams changes from regular Postgres heap tables to Iceberg tables via logical replication, ensuring sub-second data freshness. The extension exposes SQL procedures to create columnstore mirrors of Postgres tables, manage snapshots, load files, and optimize storage. It is built using the pgrx framework for Rust-based Postgres extensions and supports Postgres versions 14-18. The architecture is a hybrid of a shared library loaded into the Postgres backend and a separate background worker process. The codebase is well-structured and actively developed, though many table access method callbacks are still stubs, indicating it is in an early but functional stage.
The LLM identified 3 files worth highlighting for reuse:
Manages a tokio runtime and a Unix stream connection as static singletons using LazyLock and Mutex. This pattern is essential for bridging async Rust code with Postgres's synchronous C API. It is self-contained and can be reused in any Postgres extension that needs to perform async I/O (e.g., connecting to external services) from within a synchronous callback context.
Contains parse_table, a robust regex-based parser for SQL table names (including schema qualification and quoted identifiers), and uri_encode, a percent-encoding function compliant with AWS SigV4. Both are self-contained utilities that could be extracted and reused in any Rust project that needs to handle Postgres identifier parsing or URI encoding.
Provides the FFI interface between pg_mooncake and the DuckDB mooncake extension, including generating the init query to attach the mooncake database and retrieving the last commit LSN. This is a compact example of how to extend DuckDB's C API from within a Postgres extension, demonstrating safe CString handling and extern function calls.