$ show wildminder/ComfyUI-VoxCPM
ComfyUI-VoxCPM is a ComfyUI custom node package that provides expressive text-to-speech (TTS) using the VoxCPM model family (v1.x and v2). It supports zero-shot TTS, voice cloning (prompt or reference audio), voice design via text descriptions, LoRA inference, and LoRA fine-tuning. The architecture is a ComfyUI extension with heavy JavaScript frontend, HTTP API routes, and vendored model inference code. The core functionality is implemented in the `src/voxcpm/` directory, which contains model architectures (MiniCPM4, AudioVAE, local DiT, etc.), a standalone CLI, and training utilities. External dependencies include ComfyUI, PyTorch, Hugging Face Hub, and several optional libraries (argbind, datasets, modelscope, etc.). The codebase demonstrates good software engineering practices with clean abstractions, reusable components, and graceful degradation for optional features. However, it is tightly coupled to ComfyUI's ecosystem and vendors upstream model code, which may drift from upstream. The repository also includes a standalone `hf_progress` library for HuggingFace transfer progress tracking, which is a well-isolated, reusable component.
The LLM identified 6 files worth highlighting for reuse:
A self-contained library (v0.1.0) for tracking HuggingFace transfer progress. It defines typed ProgressEvents, supports standard and Xet (subprocess) downloads/updates with cancellation, token management, and FastAPI/SSE integration. The component is cleanly isolated, could be extracted and reused in any Python project needing robust progress reporting for HF Hub operations.
Implements a streaming AudioVAE decoder (StreamingVAEDecoder) that decodes audio chunks statefully, maintaining causal convolution buffers across calls. This is a reusable pattern for real-time audio generation where latency matters. The context manager design ensures clean state reset and is independent of the rest of the codebase.
Provides a unified interface for loading audio files and trimming silence, with fallback between torchaudio and librosa. This is a common pattern in audio ML projects and is easily extractable. The code handles edge cases (mono conversion, resampling) and uses lazy imports to avoid hard dependencies.
Implements LoRA (LoRALinear) that wraps nn.Linear and preserves state dict keys for compatibility with standard weight loading. The `apply_lora_to_named_linear_modules` function is a generic pattern for injecting LoRA into any model. This component is isolated and can be reused for other LoRA training projects.
Thin wrappers over ComfyUI's model_management for device selection, autocast, and dtype resolution. Provides `get_device()`, `get_autocast()`, and `audio_vae_dtype()` helpers that encapsulate ComfyUI-specific logic. While specific to ComfyUI, the pattern of abstracting device/dtype management is reusable in other ComfyUI custom node projects.
Defines custom ComfyUI node output types (VOICE_CLONING_CONFIG, ADVANCED_PARAMS) with default values, validation, and merge logic. This enables composable config nodes that feed into the main TTS node. The pattern of typed config objects with merge helpers is reusable for any ComfyUI custom node suite needing config composition.