PostgREST is a production-grade Haskell web server that converts an existing PostgreSQL schema into a RESTful HTTP API. At startup it introspects PostgreSQL system catalogs to build a schema cache of tables, views, relationships, routines, and media handlers. Each request is parsed into an internal AST (ApiRequest), planned against the schema cache (Plan), compiled to parameterized SQL (QueryBuilder), and executed through the vendored Hasql PostgreSQL driver. The server supports full CRUD, RPC calls, resource embedding via foreign keys, filtering, ordering, pagination, JWT authentication, OpenAPI generation, and custom PostgreSQL-defined media types. Authorization is delegated to the database: PostgREST connects as an authenticator role that does SET ROLE to the JWT-specified role, so row-level security, grants, and triggers are enforced by PostgreSQL. The system runs as a single Haskell binary with lifecycle management, SIGUSR2/config reload through LISTEN/NOTIFY, a separate admin server exposing /live, /ready, /metrics, and schema cache dumps, and metrics via Prometheus. Build and packaging is Nix-based with static linking, Docker images, and cabal/stack support. CI runs integration suites against multiple PostgreSQL versions and includes load, memory, observability, and replica tests. Maintenance is active, with frequent releases and broad production use. The codebase demonstrates a clean separation between HTTP parsing, query planning, SQL generation, and database access. It is one of the few Haskell projects at this scale with deep PostgreSQL binary-protocol integration. The vendored Hasql driver and several standalone modules—media type parsing, HTTP Range handling, SIEVE cache, JSON path parsing—are extractable as reusable components for other Haskell services.
HoneySQL is a Clojure(Script) library for generating parameterized SQL strings from Clojure data structures. It provides a clean, extensible DSL for building SQL queries programmatically, supporting multiple dialects including ANSI, MySQL, SQL Server, Oracle, PostgreSQL, and NRQL. The core API is `honey.sql/format`, which converts a hash map (e.g., `{:select [:a :b] :from [:table] :where [:= :id 1]}`) into a vector of SQL string and parameters. It also offers a helper namespace (`honey.sql.helpers`) for thread-first query construction and a registration system (`register-op!`, `register-clause!`, `register-fn!`, `register-dialect!`) for extending the DSL with custom operators, clauses, functions, and dialects. The library is designed for performance, safety (parameterized queries, limited SQL injection risks), and cross-platform compatibility (Clojure, ClojureScript, Babashka, Jolt, let-go). It is not an application or data research project; it is a pure library used as a dependency in Clojure applications that need dynamic SQL generation. The codebase is well-structured, with careful attention to edge cases (e.g., quoting, null handling, caching) and a comprehensive test suite. The extensibility mechanisms are a standout feature, allowing users to add custom SQL syntax and dialects without modifying the library's source.
This repository provides **pgrx**, a framework for writing PostgreSQL extensions entirely in Rust. It includes a CLI tool (`cargo-pgrx`) for project scaffolding, building, testing, packaging, and schema generation. The core library (`pgrx`) offers safe Rust wrappers for PostgreSQL internals: memory contexts, SPI (Server Programming Interface), arrays, triggers, aggregates, GUCs, datetime types, and more. Procedural macros (`#[pg_extern]`, `#[pg_trigger]`, `#[pg_aggregate]`, `#[derive(PostgresType)]`, etc.) automatically generate SQL DDL and Rust boilerplate, enabling a pure-Rust extension development experience. The framework supports PostgreSQL versions 13 through 19 via feature-gated FFI bindings in `pgrx-pg-sys`, which uses `bindgen` to produce per-version Rust declarations. SQL schema generation is handled at compile time by `pgrx-sql-entity-graph`, which embeds metadata in the shared library's binary sections. The architecture is modular: the `pgrx` crate re-exports public API, `pgrx-macros` provides proc macros, `pgrx-pg-sys` contains raw FFI bindings, `pgrx-sql-entity-graph` handles SQL generation, and `cargo-pgrx` is the CLI frontend. Example extensions in `pgrx-examples/` demonstrate features like custom types, background workers, logical decoding, and SPI usage. A comprehensive test suite in `pgrx-unit-tests/` includes compile-fail tests, property-based tests (via `proptest`), and integration tests that run against real PostgreSQL instances. Key external dependencies include the PostgreSQL development headers (for `pg_config` and C API), `libclang` for bindgen, and Rust ecosystem crates such as `serde`, `syn`, `quote`, `clap`, `ureq`, and `criterion` (for benchmarking). The framework is designed for extension authors; it is not a data analysis or research project.
Lumen is an open-source agent framework for chatting with data and building interactive dashboards, built on the HoloViz ecosystem (Panel, hvPlot, Bokeh). It provides a dual interface: a YAML-based declarative dashboard specification system for non-AI use cases, and an AI-powered natural language interface that generates SQL queries and Vega-Lite visualizations from user prompts. The architecture is modular, with clear separation between data sources (files, SQL databases, REST APIs, xarray), data pipelines (filters, transforms, SQL rewrites), views (hvPlot, Vega-Lite, DeckGL, tables), and an AI agent layer (Planner, Coordinator, specialized agents for SQL, visualization, validation, and source ingestion). The AI system supports multiple LLM providers (OpenAI, Anthropic, Google, Ollama, etc.), vector stores for RAG (Numpy, DuckDB, ChromaDB), and a tool system for external API integration. The codebase demonstrates high craftsmanship with reusable patterns like spec-driven component creation (MultiTypeComponent), composable SQL transforms, and a consistent agent/tool abstraction. While primarily an application framework, it includes data pipeline components that could be extracted for data engineering use. The project is actively maintained with extensive test coverage, though its large optional dependency surface and rapid feature velocity introduce some maintenance risk.
Vanna is an open-source Python framework that generates SQL queries from natural language using retrieval-augmented generation (RAG). It provides a modular agent architecture with a core `Agent` class that orchestrates LLM interactions, tool execution, and rich UI component streaming. The system supports multiple LLM providers (OpenAI, Anthropic, Gemini, Ollama, etc.) and vector store backends (ChromaDB, FAISS, Qdrant, Pinecone, etc.) for storing and retrieving training data (question-SQL pairs, DDL, documentation). It includes a comprehensive evaluation framework for comparing agent variants, a legacy Flask API for backward compatibility, and a modern Lit-based web component frontend that renders rich components (tables, charts, cards, progress bars) via SSE streaming. The architecture is highly extensible with dependency injection for lifecycle hooks, LLM middlewares, error recovery strategies, context enrichers, conversation filters, and observability providers. Key entry points include the `Agent.send_message()` async generator, the `ToolRegistry` for tool management, and server factories for Flask and FastAPI. The system relies on external dependencies for LLM APIs, vector databases, and SQL databases, all configured as optional extras. The codebase demonstrates strong engineering practices with clean abstractions, though it carries significant legacy code from a previous version and some placeholder implementations (e.g., MD5-based embeddings in Qdrant).