$ show Lux-Luna/LunaVox
LunaVox is a high-performance, cross-platform text-to-speech inference engine specialized for Qwen3-TTS models. It provides a unified C++ core with Python bindings, offering multiple interfaces: a CLI (`lunavox`), a desktop GUI (customtkinter), an HTTP/WebSocket server (FastAPI), and a Python runtime API. The architecture separates model loading, tokenization, LLM-based talker/predictor generation (via llama.cpp), and neural audio codec decoding (via ONNX Runtime) into modular components. Voice modes include base, voice cloning from reference audio, custom speaker selection, and text-described voice design. The system is designed for low-latency streaming synthesis, with benchmarks showing real-time factors below 0.2 on consumer GPUs. It includes a comprehensive model conversion pipeline from HuggingFace to GGUF/ONNX formats, a build toolchain for compiling the C++ engine, and production-ready serving with Prometheus metrics and auto-batching. The codebase demonstrates strong engineering practices with cross-platform support, structured stats contracts, and bilingual documentation, though it carries some technical debt from vendored GGUF code and platform-specific workarounds.
The LLM identified 6 files worth highlighting for reuse:
Implements TextSplitter and StreamingSentenceBuffer for language-agnostic sentence splitting using data-driven punctuation tables. Supports 10 languages with configurable strong/weak terminators and whitespace rules. The pattern is reusable for any text streaming or chunking application, with clear separation of concerns and no tight coupling to the TTS domain.
Defines SynthesisStats as a TypedDict contract shared across C++ (via ctypes), Python runtime, and HTTP server. Provides a single source of truth for performance metrics (timings, memory, VRAM) with JSON serialization. This pattern is reusable for any multi-language system requiring structured, versioned data exchange.
Implements a lazy tensor tree (LazyTorchTensor) for deferred evaluation during GGUF model conversion. Uses metaclass-based operator overloading to build computation graphs without materializing tensors, enabling memory-efficient conversion of large models. Reusable for any model export pipeline where memory is constrained.
Provides a singleton NVML monitor with dynamic symbol loading (avoids hard linking) and version-tolerant parsing of per-PID VRAM usage. Handles multiple NVML API versions and gracefully degrades if NVML is unavailable. Reusable for any application needing GPU memory tracking without direct NVML dependency.
Implements a configurable fallback chain for ONNX Runtime execution providers (CUDA, DirectML, CoreML, CPU). Builds provider options with device-specific settings and logs failures. The pattern is reusable for any ONNX-based inference system needing cross-platform provider selection with graceful degradation.
Provides a Voice class with factory methods (base, clone_file, custom, design) that encapsulate voice specification resolution. Acts as a single entry point for all voice modes, decoupling the runtime engine from voice construction. Reusable for any multi-mode synthesis system requiring a clean, extensible voice API.