PyAthena is a Python client library for Amazon Athena that implements the PEP 249 DB-API 2.0 specification with both synchronous and asynchronous execution models. It provides multiple cursor implementations (CSV, Arrow, Pandas, Polars) and an UNLOAD-based Parquet path for large-scale ETL, plus a SQLAlchemy dialect for ORM-style access. The library handles Athena's S3-backed query lifecycle, result pagination, type conversion heuristics for complex types, and the documented null-vs-empty-string semantics that differ between CSV (GetQueryResults) and Parquet (UNLOAD) paths. The architecture is a layered client: a core connection/cursor layer implementing the DB-API contract, an async sibling layer (AioCursor and related classes), and optional integration layers for pandas, Polars, Arrow, and SQLAlchemy. Key entry points are the connection factory, cursor classes, and the SQLAlchemy dialect. The library depends on boto3/botocore for AWS API calls, with optional dependencies for pyarrow, pandas, polars, and SQLAlchemy. It also ships an S3 file system abstraction (sync and async) with caching, multipart uploads, and versioning, plus a benchmark framework that spawns isolated subprocesses with an Observer pattern for SDK event monitoring, memory/thread sampling, and incremental result reporting. Security is treated seriously: the codebase maintains Trino-vs-Hive escaping distinctions for identifiers and string literals, and tests explicitly verify injection resistance. The project is actively maintained with a defined contribution workflow, CI running lint and Sphinx builds, and a worktree-based PR process. Documentation covers Athena-specific limitations including Iceberg type constraints, naming rules, and the fact that Athena does not return element-level type information for complex types, so the converter applies documented heuristics. The project uses hatch-vcs for versioning (pyathena/_version.py must never be edited manually) and supports schema-qualified queries against ENV.schema.
Redash is a full-stack business intelligence platform that enables users to connect to multiple data sources, write and schedule queries, build interactive visualizations, and create dashboards with alerting and sharing capabilities. The backend is a Python Flask application with SQLAlchemy for database access, Redis for caching and task queuing, and RQ for background job execution. The frontend is a React single-page application using Ant Design, Plotly, D3, and Leaflet, organized into pages for queries, dashboards, alerts, data sources, and administration. The system exposes a RESTful API for all operations and supports authentication via password, Google OAuth, SAML, LDAP, JWT, and API keys. The architecture follows a modular plugin pattern for query runners and visualizations. Over 35 query runners (e.g., PostgreSQL, BigQuery, Snowflake, MongoDB, Prometheus) are registered via a common base class, each implementing run_query, get_schema, and configuration_schema. The visualization library (viz-lib) is a separate package that registers chart types (line, bar, pie, map, table, cohort, sankey, etc.) with editors and renderers. Dashboards use a grid layout with widgets that can display visualizations, text, or embedded content. Alerts evaluate query results against conditions and send notifications to destinations like Slack, email, PagerDuty, and webhooks. Key data flows: users write SQL or use query runners to fetch data from connected sources; results are cached and can be visualized; dashboards aggregate multiple visualizations with parameter bindings; scheduled queries refresh results and trigger alerts. The system also supports embedding visualizations in external sites and public sharing of dashboards. While primarily an application, the extensive query runner and visualization plugin systems make it a flexible platform for data exploration and operational reporting.