ExLlamaV3 is a high-performance inference library for running large language models on consumer GPUs, with a focus on flexible quantization (EXL3 trellis format) and parallel inference. The architecture is a modular Python library with a compiled C++/CUDA extension providing custom kernels for attention (paged, MLA, DSA, recurrent), MoE routing, quantization, and sampling. Key entry points include the `Model`, `Config`, `Tokenizer`, `Cache`, and `Generator` classes, as well as CLI tools for model conversion (`convert.py`), quantization optimization (`sc_optimize.py`), and evaluation (`eval/`). Data flows: model weights are loaded from safetensors, quantized offline, then at inference time the library manages paged KV caches, supports tensor parallelism across GPUs, and provides streaming generation with filters (Formatron, LLGuidance) and custom samplers. External dependencies include PyTorch, safetensors, tokenizers, Triton, and optional packages like xformers and llguidance. The codebase demonstrates strong engineering with reusable module abstractions, a comprehensive attention dispatch system, and a sophisticated quantization pipeline, though it carries some technical drift from unmaintained dependencies and a large surface of architecture-specific files.