Vanna is an open-source Python framework that generates SQL queries from natural language using retrieval-augmented generation (RAG). It provides a modular agent architecture with a core `Agent` class that orchestrates LLM interactions, tool execution, and rich UI component streaming. The system supports multiple LLM providers (OpenAI, Anthropic, Gemini, Ollama, etc.) and vector store backends (ChromaDB, FAISS, Qdrant, Pinecone, etc.) for storing and retrieving training data (question-SQL pairs, DDL, documentation). It includes a comprehensive evaluation framework for comparing agent variants, a legacy Flask API for backward compatibility, and a modern Lit-based web component frontend that renders rich components (tables, charts, cards, progress bars) via SSE streaming. The architecture is highly extensible with dependency injection for lifecycle hooks, LLM middlewares, error recovery strategies, context enrichers, conversation filters, and observability providers. Key entry points include the `Agent.send_message()` async generator, the `ToolRegistry` for tool management, and server factories for Flask and FastAPI. The system relies on external dependencies for LLM APIs, vector databases, and SQL databases, all configured as optional extras. The codebase demonstrates strong engineering practices with clean abstractions, though it carries significant legacy code from a previous version and some placeholder implementations (e.g., MD5-based embeddings in Qdrant).
Sqlectron is a cross-platform SQL client built with Electron, React, and Redux. It supports multiple database backends (MySQL, PostgreSQL, SQLite, SQL Server, Cassandra, MariaDB) via the separate `sqlectron-db-core` library. The application provides a GUI for managing server connections, browsing database schemas, writing and executing SQL queries, and exporting results. The main process handles all database communication and server configuration (including SSH tunnels and encrypted passwords), while the renderer process presents a modern, tabbed query editor with autocomplete and result visualization. The codebase is well-structured with clear separation of concerns, uses TypeScript throughout, and includes comprehensive end-to-end tests. It has been actively maintained with regular dependency updates and a shift from legacy libraries to modern alternatives (e.g., CodeMirror, Tailwind CSS, Radix UI).
Feldera is a real-time streaming SQL analytics platform built on the DBSP (Differential Dataflow) incremental computation engine. It compiles SQL into efficient incremental circuits that process streaming data with sub-millisecond latency, supporting complex queries including joins, aggregations, window functions, and recursive queries. The system is deployed as a pipeline manager (Rust) with a REST API, a web console (Svelte), a Python SDK, a dbt adapter, and a CLI tool. It integrates with numerous data sources and sinks (Kafka, Postgres, Delta Lake, Iceberg, HTTP, NATS, Redis, etc.) and provides fault tolerance via checkpointing and storage backends. The architecture is polyglot: a Java-based SQL compiler (using Apache Calcite) translates SQL to Rust code that runs on the DBSP runtime; the pipeline manager orchestrates compilation, deployment, and lifecycle; the web console offers a full UI for pipeline management, monitoring, and profiling. The codebase also includes a Spark SQL translator (felderize) and extensive benchmark suites (Nexmark, LDBC). While primarily an application, it has strong research roots in incremental view maintenance and formal monotonicity analysis, reflected in the compiler's optimization passes and the DBSP runtime's operator design. Key reusable components include the adapter library (connector traits), the Feldera types crate (shared configs and errors), the common UI component library, and the Python client SDK. The project is actively maintained with frequent commits, comprehensive CI/CD, and a growing test suite covering SQL semantics across many data types and edge cases.
porsager/postgres is a PostgreSQL client library for Node.js, Deno, and Cloudflare Workers. It provides a full-featured, performant interface to PostgreSQL using tagged template literals for safe, parameterized queries. The library implements the PostgreSQL wire protocol from scratch with zero runtime dependencies, supporting advanced features like connection pooling, prepared statements, cursors, COPY streams, logical replication, large objects, and custom type mapping. The architecture is a factory function that returns a `sql` tagged template function, which lazily connects to the database and pipelines queries over persistent connections. The codebase is well-structured across `src/` with separate modules for connection management, query execution, protocol message building, type serialization/parsing, and error handling. It also includes transpiled builds for CommonJS, Deno, and Cloudflare Workers, each with platform-specific polyfills. The library is actively maintained, has extensive test coverage, and is published under a public domain license.
This repository is the personal blog and knowledge base of digoal, a prominent PostgreSQL DBA and database expert. It contains over a decade of blog posts (2010–2026) covering PostgreSQL internals, performance tuning, database architecture, and related topics, along with a large collection of images, PDFs, and other attachments. The blog is organized by year and month, with each post as a markdown file. The repository also includes a 'skills' directory that defines reusable AI agent skill definitions (SKILL.md files) for various tasks such as fact-checking, stock prediction, bug hunting, and course writing. These skills are structured with agent prompts, reference materials, and supporting scripts, making them extractable and reusable in AI-assisted workflows. The repo is actively maintained and serves as both a reference for database professionals and a library of AI agent capabilities.
OrientDB is a multi-model NoSQL database management system supporting document, graph, object, and relational data models within a single engine. It provides ACID transactions, a SQL-like query language, distributed clustering with automatic sharding and replication, full-text and spatial indexing via Lucene, and a Gremlin graph traversal API. The system is implemented in Java and organized into modular components: a core storage engine with pluggable index structures (SB-Tree, hash, Lucene), a binary client-server protocol, an HTTP REST API, and a web-based administration studio (AngularJS). It includes enterprise features such as backup/restore, metrics monitoring, and security auditing. The architecture follows a plugin-based server model, allowing extensions for authentication, indexing, and custom commands. External dependencies include Dropwizard Metrics, Lucene, Gremlin/TinkerPop, and various compression libraries.
sqlite-net is a lightweight .NET ORM library for SQLite, providing both synchronous and asynchronous APIs. It maps C# classes to database tables via attributes, supports CRUD operations, LINQ-like queries, transactions, migrations, encryption (SQLCipher), and backup. The library is implemented in two main source files (SQLite.cs and SQLiteAsync.cs) and uses SQLitePCLRaw or P/Invoke to interface with native SQLite. It targets .NET Standard 2.0, .NET 8, and .NET 9, and is designed for cross-platform use including Xamarin, UWP, and desktop applications. The codebase is well-structured with performance optimizations like fast column setters and prepared statement caching, and includes a comprehensive test suite. It is actively maintained with recent commits and CI integration.
This Laravel package provides an Eloquent-like interface for AWS DynamoDB, allowing developers to use familiar ORM methods with DynamoDB while maintaining compatibility with Laravel's ecosystem. The package implements a sophisticated query builder that translates Eloquent-style conditions into efficient DynamoDB operations, intelligently choosing between Query and Scan operations and selecting optimal indexes when available. Key features include support for simple and composite keys, global/secondary indexes, asynchronous operations via AWS SDK promises, query scopes, and automatic attribute marshaling/unmarshaling. The architecture follows Laravel conventions with service providers, facades, and a base DynamoDbModel class. Notable components include a condition analyzer that optimizes query operations and a parser subsystem that correctly builds DynamoDB expression syntax with proper attribute name/value handling.
This repository provides a Docker-based tool for automatically discovering and backing up databases running in Docker containers. The core logic resides in a single Python script (`db-auto-backup.py`) that uses the Docker SDK to list running containers, identifies database containers by image name (PostgreSQL, MySQL/MariaDB, Redis), and executes native backup commands (pg_dumpall, mysqldump, redis-cli SAVE). Backups are written to a configurable directory with optional compression (gzip, lzma, bz2, zstd) and can be scheduled via cron expressions using the pycron library. The tool also supports success hooks (e.g., healthchecks.io, Uptime Kuma) and environment variable configuration. It is designed to run as a standalone Docker container with access to the Docker socket, making it a lightweight, self-contained backup solution for Docker hosts. The codebase is well-structured with type hints, linting, and a comprehensive test suite that validates backup generation and compression across multiple database types.
Flyway is a database migration tool that applies version-controlled SQL scripts to manage schema changes across 50+ database engines (relational and NoSQL). It operates as a Java library, CLI, Maven/Gradle plugin, and Docker image. The core architecture centers on a plugin system where `DatabaseType`, `VerbExtension`, `ConfigurationExtension`, and `Callback` implementations are discovered via Java ServiceLoader. The migration lifecycle is orchestrated by `FlywayExectutor`: it resolves migration scripts (SQL or Java), validates them against a schema history table, applies pending migrations in transaction-aware batches, and registers the result. Telemetry and license enforcement are woven into execution. The codebase is large but well-organised into modules: `flyway-core` for the engine, `flyway-commandline` for the CLI, `flyway-database-*` for per-database parsers and connections, `flyway-nc` for a dual native-connector path, and `flyway-reports` for output generation. External dependencies include JDBC drivers, AWS SDK (S3), and third-party helpers like SQLFluff (via Docker). Key entry points are the `Flyway` API class, the `Main` CLI class, and the `VerbExtension` implementations.
Apache ShardingSphere ElasticJob is a distributed job scheduling framework providing elastic sharding, failover, misfire handling, and lifecycle management. It is a Java library that can be embedded in applications or used via Spring Boot auto-configuration. The system coordinates jobs across multiple nodes using a registry center (ZooKeeper, etcd, Nacos, or in-memory). It supports multiple job types (simple, dataflow, script, HTTP) and provides RESTful admin APIs, event tracing, and error notification. The architecture is modular with clear SPI-based extensibility for job executors, error handlers, tracing listeners, and registry center providers. The codebase is well-engineered with extensive tests, but retains some legacy integration paths (Spring XML namespace) and tight coupling to ZooKeeper in core modules.
ObjectBox Java is an open-source, embedded NoSQL database library for Java and Kotlin, targeting Android, JVM, and desktop platforms. It provides a high-performance, native-backed object persistence engine with type-safe querying, reactive data observation, object relations (ToMany/ToOne), vector search via HNSW indexes, and optional client-server or mesh peer-to-peer synchronization. The code is organized into multiple Gradle modules including a core `objectbox-java` module with JNI bindings, an annotation API (`objectbox-java-api`), Kotlin extensions, RxJava 2/3 adapters, Android-specific integrations (LiveData, Admin web UI, Paging DataSource), and a mesh sync module using Google Nearby Connections. The architecture is library-centric with a clear separation between Java-facing API, internal serialization (vendored FlatBuffers), and native runtime. Extensive unit and integration tests cover core CRUD, transactions, queries, relations, sync, and error handling. The repository is actively maintained with a version 6.0.0-beta, publishes to Maven Central, and uses multi-platform CI (GitLab, Jenkins). While the codebase includes some legacy APIs and generated FlatBuffers code, the core abstractions exhibit solid software engineering practices with reusable patterns for reactive subscriptions, property converters, and cross-platform native library loading.
ListenBrainz Server is the backend and frontend for ListenBrainz, an open-source music listening history platform operated by the MetaBrainz Foundation. In production it ingests listening events from multiple sources (Spotify, Last.fm, Libre.fm, Maloja, file uploads, and direct API submissions), stores them in TimescaleDB, enriches them with MusicBrainz identifiers via an MBID mapping pipeline, and exposes them through a public REST API, a Last.fm-compatible API, WebSocket streams, and a React single-page application. The system also computes per-user and site-wide statistics, artist/recording/user similarity, collaborative-filtering recommendations, and annual 'Year in Music' summaries using Apache Spark batch jobs orchestrated through RabbitMQ and HDFS. The architecture is a hybrid of a traditional Flask web application and a set of background data-processing services. The main web server handles user authentication, API endpoints, playlist management, social features, and page rendering, while separate workers consume RabbitMQ queues for timescale writes, Spark job requests, metadata cache crawling (Spotify, Apple Music, SoundCloud, Internet Archive), and background imports/exports. The frontend is a large TypeScript/React application with React Router, TanStack Query, Jotai state, and a custom multi-source audio player called BrainzPlayer. Although the repository is primarily an application, it contains substantial data-engineering machinery—Spark incremental stats engines, dump generation/import, MBID canonicalization, and similarity algorithms—making it a genuinely hybrid codebase with the application character dominant. The codebase is well-structured for its size: it has clear module boundaries, abstract base classes for importers and OAuth services, a reusable incremental stats query-provider pattern, and extensive unit and integration tests. However, it also carries significant complexity and technical debt: multiple database backends, duplicated SQL across Spark entity stats, legacy compatibility endpoints, hardcoded external URLs, and some skipped or heavily mocked tests. The project is actively maintained, with a steady stream of schema migrations and feature additions through 2026.
This repository implements a modern Automatic Number Plate Recognition (ANPR) pipeline as a production-ready application. It ingests images (via HTTP upload or WebSocket stream), runs detection using the `fast-alpr` library (YOLO-based ONNX model), performs OCR with `fast-plate-ocr`, applies post-processing (confusion-pair correction, Turkish plate format parsing, temporal majority voting), optionally tracks vehicles via an IoU-based multi-object tracker, and stores only HMAC-SHA256 hashed plate texts for privacy compliance. The system is exposed through a FastAPI REST API with WebSocket support and a HTMX web UI, plus a Typer CLI for inference and server management. It is containerized with Docker and publishes to PyPI and GHCR. The repository also contains a legacy directory (`legacy/`) with a complete YOLOv5 training pipeline for license plate detection, but that code is explicitly unmaintained and serves as historical reference only. The modern codebase is actively maintained, well-tested, and demonstrates strong software engineering practices including protocol-based abstractions, structured logging, Prometheus metrics, and privacy-by-design persistence.
Chat2DB is an AI-powered database client and SQL workspace supporting over 40 databases via a plugin architecture. It provides a full-featured SQL editor with syntax highlighting, autocompletion, and execution, along with database metadata browsing, data import/export, BI dashboards, and an AI assistant that can generate SQL, explain queries, and create charts. The system is designed as a hybrid desktop/web application, with a React/TypeScript frontend (using Ant Design, Monaco Editor, Zustand for state management) and a Java Spring Boot backend with a layered domain model. The desktop version uses JCEF (Chromium Embedded Framework) for native integration, while the web version runs as a standalone server. Database support is modular via SPI plugins, each providing metadata, SQL builders, value processors, and completion logic. The architecture is centered around a plugin system where each database type implements interfaces like IPlugin, IDbMetaData, and IDbManager, enabling extensible multi-database support. The frontend uses Zustand stores with slice-based state management, and the backend provides REST APIs for SQL execution, AI chat, and data management. Key entry points include the frontend main page (CommunityMainPage), backend controllers (e.g., DbSqlController, AiChatController), and plugin registration via META-INF/services. Data flows involve SQL execution requests from the editor to the backend, which delegates to the appropriate database plugin and returns results asynchronously with streaming support for AI and large result sets. External dependencies include Java/Spring Boot, React, Monaco Editor, ANTLR for SQL parsing, ECharts for charts, and various JDBC drivers for database connectivity. The system also integrates with AI model APIs (e.g., OpenAI, Claude) via configurable endpoints. The codebase demonstrates a strong separation of concerns through its SPI plugin pattern and state management slices, though it exhibits some duplication across database plugins and has limited test coverage.
nedb-promises is a lightweight Node.js library that wraps the @seald-io/nedb embedded database (a fork of nedb) with a promise-based API. It provides a Datastore class that extends EventEmitter, offering methods like find, insert, update, remove, and count that return Promises or thenable Cursor objects. The Cursor class supports fluent chaining (sort, skip, limit, project) and can be awaited directly. The library also includes MongoDB-compatible aliases (insertOne, updateMany, deleteOne, etc.) and a static `create()` method that returns a Proxy to access the underlying nedb instance's properties. The codebase is clean, well-tested, and leverages modern JavaScript features (async/await, Proxy, EventEmitter). It is a classic promise wrapper with a thoughtful event system for operation success and error broadcasting.
Fireproof is a local-first, real-time document database with CRDT-based conflict resolution, end-to-end encryption, and multi-platform support (browser, Node.js, Deno, Cloudflare Workers). The monorepo contains the core database engine (`@fireproof/core`), a cloud sync backend (Hono + Durable Objects), a dashboard web application for tenant/ledger management, a CLI tool for build and DevOps tasks, and a React hooks library (`use-fireproof`). The architecture is modular with clear separation between storage gateways (IndexedDB, filesystem, memory, cloud), key management (KeyBag with fingerprinting), device identity (X.509-like PKI), and protocol layers (WebSocket/HTTP message dispatch). The codebase exhibits strong engineering practices including extensive TypeScript usage, comprehensive testing (though some tests are skipped or known to fail), and reusable patterns like the Gateway interceptor, Result type, and Evento command pipeline. However, complexity is high due to multiple runtime targets and active refactoring between versions, and there are known critical bugs (e.g., remote sync not triggering subscriptions in React hooks).
The ReplicaDB repository is a high-performance, open-source command-line tool for bulk data replication between heterogeneous databases. It supports a wide range of data sources, including relational databases, NoSQL databases, data virtualization platforms, file formats, cloud storage, and streaming platforms. The tool is cross-platform compatible with Windows, Linux, and macOS, and leverages parallel data transfer for optimal performance and system utilization during large-scale data migrations and synchronization tasks.
Sled is an embedded key-value store written in Rust that provides ACID transactions, crash safety, and high concurrency. It uses a lock-free architecture based on a lock-free pagecache, lock-free log, and lock-free B+ tree with epoch-based memory reclamation. The database supports multiple keyspaces, watch prefixes, merge operators, and a crash-safe monotonic ID generator. Currently in active development with a major rewrite underway (komora project), it is alpha software with known instability and frequent breaking changes, but demonstrates advanced concurrent data structure techniques.
Redka is a Redis-compatible data store implemented in Go, using SQLite or PostgreSQL as the storage backend. It reimplements core Redis data structures (strings, lists, sets, hashes, sorted sets) with full TTL support, key scanning, and set/sorted-set aggregation operations. The system can be deployed either as an embedded Go library (via the `redka` module) or as a standalone RESP-compatible server (single binary or Docker image). Architecture follows a clean layered design: a top-level `DB` struct provides transactional access to per-type repositories (e.g., `ZSet()`, `Hash()`), which in turn use a dialect abstraction layer (`sqlx`) to generate SQLite- or PostgreSQL-specific queries. The RESP server is built on `tidwall/redcon` with a command dispatch pattern and a reusable parser combinator framework for argument parsing. While feature-complete for core Redis operations, the project is explicitly in maintenance mode—no new features are planned, and documentation notes incomplete command coverage (e.g., blocking operations, some sorted set commands). The codebase is well-structured with extensive test coverage, but Postgres support shows signs of drift (many queries aliased to SQLite) and the command dispatch switch has grown large.