Full MySQL-compatible SQL parser generated by goyacc with extensible AST. Used by external projects (Bytebase, SQLC, Bytebase, Gaea). Supports DDL, DML, hints, CTEs, window functions, JSON, vector types. Clean separation from TiDB internals.

Unified object storage abstraction supporting S3, GCS, Azure Blob, local, HDFS, memory with consistent interface. Features: retry logic, multipart upload, compression (gzip/snappy/zstd), encryption, batch operations, recording/replay for testing, mock implementations. Used by BR, Lightning, Dumpling.

Table/schema filtering library with fnmatch wildcards, regex, blocklist, file import, quoted identifiers, escape chars. Precedence rules (last match wins). Used across all TiDB ecosystem tools (BR, Lightning, Dumpling, CDC, TiCDC).

Distributed eXecution Framework (DXF) for long-running distributed tasks. Components: proto (task/subtask/step definitions), scheduler (owner-side admission, slot reservation, preemption), taskexecutor (node-side execution, preemption, meta recovery), planner (logical-to-physical), storage (persistent task state), handle (submit/wait/pause/resume/cancel). Used by IMPORT INTO and DDL distributed backfill.

Distributed timer framework for periodic tasks with cron/interval scheduling, persistent storage (TiDB system tables), hook interface (OnPreSchedEvent/OnSchedEvent), watermark for recovery, exactly-once semantics via event closure. Supports multiple runtimes with distributed locking.

Extension framework for integrating standalone code with TiDB. Supports: custom auth plugins (with client-side plugin mapping), session event hooks (connection/statement), custom system variables, dynamic privileges, custom functions, bootstrap SQL, access check overrides. Factory-based dynamic registration based on config.

High-performance columnar chunk processing with: mutable/immutable rows, column vectors (int/float/string/decimal/time/json), spill-to-disk for large sorts/joins, multiple codecs (chunk/row), memory arena allocation, zero-copy pooling, iterator framework, row containers for hash aggregation/join.

Expression evaluation framework with generated vectorized implementations for SIMD. 100+ builtin functions (math, string, time, JSON, regex, encryption, bit, control, cast, comparison). Code generator produces type-specialized kernels. Constant folding/propagation. Chunk-based execution. DistSQL pushdown support.

Cost-based optimizer with dual engines: traditional rule-based (RBO) and Cascades framework. Features: plan cache (instance/global), parameterized plans, CBO with cardinality estimation (histograms, CMS, FM-sketch), join reorder (DP/greedy), predicate pushdown, partition pruning, MPP enforcement, index merge, lateral join, CTE optimization, plan replay capture.

Online DDL execution with job-based architecture: SQL → Job → Owner schedules → Workers execute steps (meta change → reorg → public). Distributed backfill via DXF for add-index. Schema versioning with etcd-based sync. Multi-schema change support. Pause/resume. Fast path for metadata-only changes. Foreign key, partition, TTL, generated column support.

Auto-analyze priority queue with sophisticated priority calculation: considers table size, change rate, time since last analyze, partition type. Heap-based with dynamic/static partitioned table jobs. DDL-aware queue maintenance. Calculator analysis test with golden file validation across 1K-100M row scenarios.

Complete collation support: Unicode 9.0.0 (uca0900_ai_ci), Unicode 4.0.0, GBK, GB18030, binary, pinyin. Generated weight tables from UCA allkeys.txt. Code generator produces optimized comparison functions. Supports CI/AI variants. Used in sorting, comparison, indexing.

Global sort engine for TB-scale data import: encodes SQL rows to KV pairs, sorts via external merge sort (multi-way merge, spill management), detects duplicates across engines, computes checksums, writes SST files for TiKV ingestion. Merge v2 with improved memory management.

External sorting library: multi-way merge with priority queue, disk-based sorter with configurable buffer, parallel sort workers with spill helper, Top-N optimization with chunk heap, sort spill test coverage. Used by global sort, hash join spill, window functions.

Memory arena allocator for reduced GC pressure: bump-pointer allocation from large chunks, thread-local caches, automatic chunk management, benchmark showing significant GC reduction. Used in chunk processing, expression evaluation, sort/join operators.

Multi-version map for MVCC: FNV hash, per-key version chains, snapshot isolation reads, concurrent writes with lock striping, background GC of old versions, memory usage tracking. Used in transaction layer for versioned key-value storage.

Parquet reader/writer with Apache Arrow: Spark legacy datetime/decimal rebase handling, schema builder with type mapping, column buffers for efficient encoding/decoding, writer behavior tests, test files from Hive/Aurora. Used by Lightning for Parquet import.