$ show ClickHouse/clickhouse-connect
clickhouse-connect is a production-grade, high-performance Python database driver for ClickHouse that provides synchronous and asynchronous client APIs, a full DB-API 2.0 interface, and a comprehensive SQLAlchemy dialect with Alembic migration support. The architecture separates transport backends (HTTP, embedded chDB) behind a clean protocol contract (`SyncBackend`/`AsyncBackend`) and offers an optional Rust native codec (`_ch_core`) for critical encoding/decoding paths. Key entry points are `clickhouse_connect.get_client()` (sync) and `clickhouse_connect.get_async_client()`, which return `Client`/`AsyncClient` instances. These expose methods for SQL queries, inserts, commands, streaming, and direct integration with Pandas, NumPy, Apache Arrow, and Polars via dedicated result-type methods (e.g., `query_df`, `query_arrow_stream`). The driver handles ClickHouse-specific features including multiple wire formats (Native, RowBinary, Arrow, Parquet, JSONEachRow), compression (lz4, zstd, brotli, gzip), external data attachments, server-side parameter binding, and advanced data types (JSON, Variant, Dynamic, QBit, Decimal, DateTime64, temporal types with timezone control). External dependencies include `urllib3` (sync HTTP), `aiohttp` (async), `pyarrow` (optional), and the ClickHouse Rust codec wheel (optional but recommended for performance). The codebase is well-tested with both unit and integration tests against live ClickHouse servers, including TLS and proxy configurations.
The LLM identified 5 files worth highlighting for reuse:
Defines `SyncBackend` and `AsyncBackend` protobuf-like protocols using type annotations. This clean interface separates transport logic from client logic, making it trivial to add new backends (e.g., gRPC, custom HTTP) without modifying the client API. It is self-contained and could be extracted as a reusable pattern for any database driver.
Implements `AsyncSyncQueue` that bridges an async producer to a sync consumer (and reverse) with bounded queues and shutdown support. Includes `sync_q` and `async_q` properties for clean backpressure. This is a reusable cross-thread/coroutine communication primitive that can be extracted and used in any mixed-sync/async streaming application.
Provides timezone resolution utilities: `normalize_timezone`, `is_utc_or_fixed`, `utcfromtimestamp` (with epoch arithmetic), and `safe_tzinfo`. These handle DST, Etc/UTC quirks, and PEP 495 fold-ambiguity. The module is dependency-free (uses stdlib `zoneinfo`/`pytz`) and is reusable by any Python project needing robust timezone handling.
Contains reusable custom SQLAlchemy clause elements: `ArrayJoin`, `ClickHouseJoin`, `Lambda`, `PreWhereClause`, `LimitByClause`, and `_JSONSubcolumn`. Each is a self-contained `ClauseElement` subclass with proper `cache_key` support for SQLAlchemy 2.0. These demonstrate how to extend SQLAlchemy's expression language for a specific database and can be adapted for other dialects.
Implements `AsyncSyncQueue`-based streaming classes: `StreamingResponseSource`, `ReadAheadSource`, and `StreamingInsertSource`. These convert async generators into sync iterators (and vice versa) with bounded queues and thread-backed read-ahead. The pattern is transport-agnostic and can be reused in any data pipeline that needs to bridge asyncio and threading contexts.