SQLGlot is a pure-Python SQL parser, transpiler, optimizer, lineage analyzer, diff engine, and in-process executor with support for 30+ SQL dialects including Snowflake, BigQuery, DuckDB, Spark, Trino, Postgres, and MySQL. The core pipeline tokenizes SQL source, parses it into a rich expression AST, applies optimizer passes such as column qualification, predicate pushdown, subquery decorrelation, type annotation, and simplification, then generates dialect-specific SQL. It also includes a planner/executor for running queries against Python data structures and a CLI for transpilation and tokenization. Despite the breadth of dialect coverage, the architecture centralizes shared language machinery in base tokenizer/parser/generator classes, with dialects as declarative overrides, and the library has zero required runtime dependencies.
DBeaver is a universal database management tool and SQL client, built as a desktop application on the Eclipse RCP (OSGi plugin) framework. It provides a rich graphical interface for connecting to over 100 database systems via JDBC/ODBC drivers, including relational databases (MySQL, PostgreSQL, Oracle, SQL Server, SQLite, etc.), cloud databases (BigQuery, Athena, Snowflake), and NoSQL systems (MongoDB, Cassandra) in commercial editions. The application offers a full suite of database development features: a multi-tab SQL editor with syntax highlighting, code completion, and execution plan visualization; a data editor with grid, form, and GIS geometry views; ER diagram designer; data export/import wizards supporting CSV, JSON, XML, SQL, Markdown, and Excel; dashboard widgets for query-based monitoring; and an integrated AI assistant (OpenAI, Copilot) for natural language query generation and schema analysis. The architecture follows a strict model/UI separation enforced by OSGi plugin boundaries. The core model plugin (`org.jkiss.dbeaver.model`) defines interfaces for database objects, data values, execution contexts, and security, which are implemented by database-specific extension plugins (`org.jkiss.dbeaver.ext.*`). UI plugins (`org.jkiss.dbeaver.ui.*`) provide editors, navigators, and dialogs, while data transfer, debugging, and AI features are isolated in their own plugins. This design allows the model layer to be reused in the web-based CloudBeaver product. Key entry points include the `DBeaverApplication` class for startup, the `SQLEditor` for query editing, and the `DataTransferWizard` for import/export. External dependencies include the Eclipse RCP framework, JDBC drivers, ANTLR4 for SQL grammar parsing, JSQLParser for SQL analysis, and various third-party libraries for SSH tunneling, AWS SDK, and AI API communication. Data flows center around database connections: users configure data sources via connection wizards, execute SQL queries through the editor, and view results in data grids or export pipelines. The data transfer subsystem implements a producer/consumer pattern with pluggable transformers and serializers, supporting streaming for large datasets. The AI integration uses a provider abstraction to interface with different LLM APIs, generating SQL from natural language and providing schema-aware context. The repository also includes a CLI tool (`dbvr`) for CI/CD automation and a headless test platform. While the community edition is fully open-source, commercial versions add NoSQL support, additional AI providers, and premium features, with some code paths guarded by feature flags.
Redash is a full-stack business intelligence platform that enables users to connect to multiple data sources, write and schedule queries, build interactive visualizations, and create dashboards with alerting and sharing capabilities. The backend is a Python Flask application with SQLAlchemy for database access, Redis for caching and task queuing, and RQ for background job execution. The frontend is a React single-page application using Ant Design, Plotly, D3, and Leaflet, organized into pages for queries, dashboards, alerts, data sources, and administration. The system exposes a RESTful API for all operations and supports authentication via password, Google OAuth, SAML, LDAP, JWT, and API keys. The architecture follows a modular plugin pattern for query runners and visualizations. Over 35 query runners (e.g., PostgreSQL, BigQuery, Snowflake, MongoDB, Prometheus) are registered via a common base class, each implementing run_query, get_schema, and configuration_schema. The visualization library (viz-lib) is a separate package that registers chart types (line, bar, pie, map, table, cohort, sankey, etc.) with editors and renderers. Dashboards use a grid layout with widgets that can display visualizations, text, or embedded content. Alerts evaluate query results against conditions and send notifications to destinations like Slack, email, PagerDuty, and webhooks. Key data flows: users write SQL or use query runners to fetch data from connected sources; results are cached and can be visualized; dashboards aggregate multiple visualizations with parameter bindings; scheduled queries refresh results and trigger alerts. The system also supports embedding visualizations in external sites and public sharing of dashboards. While primarily an application, the extensive query runner and visualization plugin systems make it a flexible platform for data exploration and operational reporting.