ComfyUI-VoxCPM is a ComfyUI custom node package that provides expressive text-to-speech (TTS) using the VoxCPM model family (v1.x and v2). It supports zero-shot TTS, voice cloning (prompt or reference audio), voice design via text descriptions, LoRA inference, and LoRA fine-tuning. The architecture is a ComfyUI extension with heavy JavaScript frontend, HTTP API routes, and vendored model inference code. The core functionality is implemented in the `src/voxcpm/` directory, which contains model architectures (MiniCPM4, AudioVAE, local DiT, etc.), a standalone CLI, and training utilities. External dependencies include ComfyUI, PyTorch, Hugging Face Hub, and several optional libraries (argbind, datasets, modelscope, etc.). The codebase demonstrates good software engineering practices with clean abstractions, reusable components, and graceful degradation for optional features. However, it is tightly coupled to ComfyUI's ecosystem and vendors upstream model code, which may drift from upstream. The repository also includes a standalone `hf_progress` library for HuggingFace transfer progress tracking, which is a well-isolated, reusable component.
Fish Speech is a production-grade multilingual text-to-speech system with a dual-autoregressive transformer architecture (4B parameter slow AR for temporal semantics, 400M fast AR for residual codebooks). It supports real-time voice cloning from short reference audio (10-30s), multi-speaker multi-turn generation, and fine-grained prosody/emotion control via natural language tags. The system is deployed through multiple interfaces: a Gradio web UI, a React-based modern web UI, a production ASGI API server (Kui) with streaming support, and command-line tools for inference and batch processing. Its architecture is modular: a TTSInferenceEngine orchestrates reference loading, LLM inference, and VQ decoding; an RVQ audio codec (10 codebooks, ~21 Hz frame rate) compresses audio; and the dual-AR model is structurally isomorphic to standard LLMs, enabling reuse of inference optimizations like paged KV cache and RadixAttention. The repository also includes training infrastructure (Lightning + Hydra) with LoRA fine-tuning, protobuf-based dataset pipelines, and a comprehensive i18n framework. While primarily a deployable application, it retains strong research character with published papers, benchmark results, and custom model implementations.
WanAndroid is a polished, feature-rich Android client for the WanAndroid platform (wanandroid.com), a Chinese community for Android developers. The application provides comprehensive browsing of articles, projects, WeChat articles, knowledge trees, navigation, Q&A, book tutorials, and daily news aggregation from multiple Chinese sources. It implements a classic MVP architecture with a modular project structure (app, basic/core, basic/ui, basic/utils, basic/l10n, component/ad). Key entry points include SplashActivity and MainActivity (hosting a bottom-tab fragment container). The system heavily leverages OkHttp/RxHttp for network requests to the WanAndroid REST API, DiskLruCache for offline caching, Room for local persistence (read-later and read-history), and EventBus for cross-component events. Beyond vanilla browsing, it features a custom WebView subsystem (WebInstance, WebContainer) with resource interception, offline HTML/ CSS caching, a reading mode using readability4j and Jsoup, and JavaScript injection for image clicks. It also includes text-to-speech (TTS) via Android TTS engine with AI-powered summarization, biometric quick login, QR code scanning, card/article sharing, a coin/ranking gamification system, and in-app updates. External dependencies include Google AdMob for ads, optional AI provider APIs (e.g., DeepSeek), and the WanAndroid server endpoints. The codebase is a mix of Java and Kotlin, with several reusable custom UI components like BottomDrawerLayout, ImagePreviewView, CollectView, and custom pull-to-refresh headers with Chinese poetry.
VoxCPM is a tokenizer-free, multilingual text-to-speech (TTS) system developed by OpenBMB. It supports voice cloning, voice design, and controllable speech generation across 30 languages and 9 Chinese dialects, outputting 48kHz audio. The architecture is a diffusion-autoregressive model built on MiniCPM-4, comprising AudioVAE, Local Encoder, Text-Semantic LM, Residual Acoustic LM, and Local DiT. The repository provides multiple interfaces: a Gradio web application (app.py) for interactive use, a command-line interface (voxcpm.cli) for batch processing, a fine-tuning web UI (lora_ft_webui.py) for LoRA and full-parameter training, and Python APIs for integration. It also includes utilities for audio enhancement (ZipEnhancer), timestamp alignment, and text normalization. While primarily an application, it contains research-grade model definitions and training pipelines, making it a hybrid project with a strong application focus.
MockingBird is a Chinese voice cloning system implementing the SV2TTS (Speaker Verification to Text-to-Speech) framework. It provides a complete pipeline for few-shot voice cloning: a speaker encoder extracts d-vector embeddings from reference audio, a synthesizer (Tacotron with Global Style Tokens or VITS) converts text to mel-spectrograms conditioned on the speaker embedding, and a vocoder (HiFi-GAN, Fre-GAN, or WaveRNN) generates waveforms. It also includes a PPG-based voice conversion path using a Conformer-based PPG extractor and a PPG2Mel converter. The system supports multiple Chinese datasets (aidatatang_200zh, magicdata, aishell3, data_aishell) and provides web UIs (Streamlit/FastAPI), a PyQt5 toolbox, and CLI tools for training and inference. The codebase is a fork of Real-Time-Voice-Cloning adapted for Mandarin with significant additions including VITS synthesizer, multiple vocoder options, emotion extraction via wav2vec2, and a modern web API layer. While the original author has moved to a commercial product (noiz.ai), the repository shows recent commits (2026) and maintains a large community (36k stars).
Amphion is a comprehensive open-source research toolkit for audio, music, and speech generation, developed by OpenMMLab. It covers multiple generative tasks including text-to-speech (TTS), singing voice conversion (SVC), voice conversion (VC), text-to-audio (TTA), neural audio codecs, and vocoders. The codebase is organized around config-driven experiments (JSON + shell scripts) with modular base classes for trainers, datasets, and inference. It ingests a wide variety of public datasets (LJSpeech, LibriTTS, VCTK, M4Singer, Emilia, etc.) and provides extensive preprocessing pipelines for feature extraction (mel-spectrograms, F0, energy, phonetic alignments, self-supervised features from HuBERT/Whisper). The primary research focus is on high-quality, zero-shot, and controllable generation using architectures like diffusion models, flow matching, GANs, autoregressive transformers, and vector-quantized codecs. The repository also includes evaluation metrics (F0, energy, intelligibility, speaker similarity) and visualization tools (SingVisio). While primarily a research codebase, it includes inference scripts and Gradio demos for practical use. External dependencies include PyTorch, HuggingFace Transformers, WeNet, Descript Audio Codec, and various pretrained models.
This repository contains a collection of Advanced Combat Tracker (ACT) plugins designed to enhance Final Fantasy XIV gameplay. The main components include SpecialSpellTimer for ability cooldown tracking, UltraScouter for combat information overlays, TTSYukkuri for text-to-speech alerts, and XIVLog for gameplay video capture. The architecture follows a modular plugin pattern with shared functionality in FFXIV.Framework, utilizing WPF and MVVM for UI components. The system integrates with game memory reading via Sharlayan, processes combat logs for real-time event detection, and provides user-configurable overlays and alerts.
FastRTC is a Python library that enables developers to build real-time audio/video streaming applications with minimal code. It provides a high-level `Stream` class that integrates with Gradio for automatic UI generation and supports both WebRTC and WebSocket protocols. The library includes built-in handlers for voice activity detection (VAD) via Silero, speech-to-text via Moonshine, text-to-speech via Kokoro/Cartesia, and turn-taking logic. It also offers TURN credential helpers for Cloudflare, Twilio, and Hugging Face. The architecture revolves around defining a handler function that processes media chunks, wrapping it with turn-taking decorators, and creating a Stream object that handles signaling, connection management, and UI rendering. The codebase is actively maintained with frequent commits and an extensive set of demos integrating with various LLMs and AI services.
espeak-ruby is a Ruby gem providing a simple API to generate Text-to-Speech audio using the `espeak` command-line tool and optionally compress to MP3 using `lame`. It exposes two main classes: `ESpeak::Speech` for generating speech (speak, save to file, or retrieve raw bytes) and `ESpeak::Voice` for listing available voices. The architecture is a thin wrapper around external CLI commands using `IO.popen`, with a clean hash-based options interface for pitch, speed, voice, amplitude, and capital letter emphasis. The repository also includes a full demo web application (Roda + Stimulus + WaveSurfer) that demonstrates the library in a browser, with a Kubernetes deployment configuration. The core library is well-structured, tested, and actively maintained, but it is a straightforward wrapper with no complex internal logic.
GPT-SoVITS is a production-grade text-to-speech and voice conversion system with a Gradio-based WebUI and FastAPI REST API. It supports zero-shot voice cloning from a 5-second reference audio and few-shot fine-tuning with 1 minute of data. The system integrates multiple model versions (v1–v4, v2Pro, v2ProPlus) with autoregressive (GPT-style) and non-autoregressive (SoVITS/CFM) components, plus a BigVGAN vocoder. It includes a full pipeline for dataset preparation (voice separation, ASR, text labeling), training (with DDP, LoRA, DPO), and inference (streaming, parallel, ONNX/TorchScript export). The codebase is heavily modular with reusable text frontends for Chinese, Japanese, Korean, Cantonese, and English, as well as audio processing tools (UVR5, BS-Roformer, AP-BWE). Despite its breadth, the code exhibits significant technical debt: version sprawl, hardcoded paths, global state, and experimental hacks. It is actively maintained with frequent bug fixes and feature additions.
This repository provides the official PyTorch implementation of TaDiCodec, a text-aware diffusion speech tokenizer accepted at NeurIPS 2025. The core contribution is a novel speech codec that achieves ultra-low bitrate (0.0875 kbps at 6.25 Hz) by combining a non-autoregressive Llama encoder, binary spherical quantization (BSQ), and a flow-matching diffusion decoder conditioned on text and prompt speech. The repo also includes token-based zero-shot TTS models: an autoregressive (AR) model using LLMs (Qwen2.5, Phi-3.5) and a masked diffusion (MGM) model. Inference pipelines with automatic model downloading from Hugging Face are provided. Training scripts for the tokenizer are partially implemented (dataloader, trainer class), but TTS model training scripts are incomplete. The project has migrated to a new repository (AmphionTeam/Diffusion-Speech-Tokenizer), and this repo is no longer the primary development branch, though recent commits indicate some ongoing activity.
KEA Research is a full-stack multi-provider AI research chat application. The backend is a FastAPI service backed by SQLite/SQLAlchemy, exposing chat, admin, health, user, and TTS endpoints. The frontend is an Astro + TypeScript single-page app that stores chat history and pipeline data in IndexedDB, with a Bootstrap-based UI. Deployment is containerized via Docker Compose with nginx handling TLS and SSE-friendly proxying. The core product logic is a four-step research pipeline: initial multi-provider responses, Mixture-of-Agents refinement, peer evaluation, and a final KEA synthesis with facts, consensus, flagged claims, rankings, and confidence scores. A web admin panel manages providers, models, provider sets, users, and app settings, while the frontend supports research layers, bookmarks, notes, drafts, export, attachment ingestion, and text-to-speech via a Kokoro ONNX model.
A cross-platform media aggregation desktop and mobile application built with Tauri v2 and Vue 3. It allows users to create, import, and manage custom JavaScript-based source extensions that scrape or fetch content from websites for books, comics, videos, photos, and music. The app includes a built-in code editor (Monaco) for authoring extensions, a download manager with HTTP and M3U8 support, text-to-speech (Edge TTS), cloud synchronization with conflict resolution (tombstones, incremental sync), DLNA casting (Android), and a source marketplace. The architecture is plugin-heavy, with many custom Tauri plugins for web scraping, media session, and command bridges. While the core functionality is comprehensive and actively maintained, the codebase exhibits tight coupling to custom plugins, limited test coverage, and security concerns from dynamic code execution of user-provided source scripts.
This repository is a small Python demo application that turns a webcam feed into a real-time David Attenborough-style nature documentary narration. It consists of two independent processes: capture.py grabs frames from the webcam every 2 seconds, resizes them to a maximum dimension of 250 pixels, and writes them to frames/frame.jpg; narrator.py polls that file, base64-encodes it, sends it to OpenAI's GPT-4 Vision API with a snarky Attenborough system prompt, appends the response to a running conversation script, synthesizes speech via ElevenLabs, and plays the audio aloud. The two processes communicate through the filesystem, which is a simple but effective decoupling pattern for a demo. The code is functional and readable, with a few thoughtful touches: the image is downscaled before upload to reduce API cost and latency, and the narrator retries reading the frame file when it catches a transient EACCES error caused by the capture process writing concurrently. However, it is a proof-of-concept rather than a production application: there are no tests, no error handling for API failures, hardcoded paths, and the pinned dependencies are from late 2023. The OpenAI model 'gpt-4-vision-preview' and the ElevenLabs 'generate/play' API are both deprecated, so the code would need updates to run against current APIs. Despite a recent commit timestamp in the metadata, the codebase is effectively a frozen 2023 demo. The maintainer has not updated dependencies or addressed the deprecated APIs, and 41 open issues remain. It is best treated as a reference implementation or a starting point for a webcam-to-AI-narration pipeline, not as a maintained library.
Bert-VITS2 is a multilingual text-to-speech system based on VITS2 (Variational Inference Text-to-Speech with improved Transformer backbone). It supports Chinese, Japanese, and English synthesis with optional emotion conditioning via pre-trained models (WavLM, wav2vec2-emotion, CLAP). The codebase provides a complete pipeline: data preprocessing (resampling, text normalization, BERT feature extraction), multi-speaker training with distributed support, inference through a Gradio web UI or FastAPI server (hiyoriUI.py), and ONNX export for deployment. The architecture uses separate per-language BERT models for text encoding, a flow-based decoder, stochastic duration predictor, and multi-scale discriminators (including a WavLM-based discriminator). The repository includes numerous versioned subdirectories (oldVersion/, onnx_modules/) for backward compatibility with model versions 1.0.1 through 2.4, resulting in significant code duplication and maintenance overhead. The README explicitly states the project is no longer maintained and recommends the successor project Fish-Speech.