Gradio is a full-stack Python library and web framework for rapidly creating interactive web UIs for machine learning models, APIs, or any Python function. It provides a high-level Interface class for simple demos, a flexible Blocks API for custom layouts and event-driven apps, and a ChatInterface for chatbot applications. The backend is built on FastAPI and Uvicorn, serving both a REST API and WebSocket-based streaming. The frontend is a Svelte 5 / TypeScript application with per-component packages, a SvelteKit SSR app, and a standalone SPA that can be embedded via a custom HTML element. The library includes a rich component library (30+ input/output widgets), layout primitives (Row, Column, Tabs, Accordion, Sidebar), theming support, internationalization, and a CLI for development, deployment, and custom component creation. It also provides official Python and JavaScript client libraries for programmatic access to Gradio apps, and supports MCP (Model Context Protocol) for tool/resource integration. The repository contains over 100 runnable demos covering a wide range of ML use cases (image classification, text generation, speech recognition, object detection, chatbots, etc.), many of which integrate with Hugging Face models and datasets. While primarily an application framework, the extensive demo collection and focus on ML model visualization give it a secondary data/research character.
MinerU is a production-grade document parsing engine that converts PDFs, images, and Office documents (DOCX, PPTX, XLSX) into structured Markdown and JSON. It supports multiple backends: a fast pipeline (layout analysis + OCR + formula/table recognition), a high-accuracy hybrid mode combining VLM and OCR, and a pure VLM mode using vLLM/LMDeploy. The system is deployed as a CLI tool, FastAPI service, Gradio web UI, or Docker container, with integrations for RAG frameworks like LangChain, LlamaIndex, and Dify. The codebase is well-architected with clean abstractions for data I/O (file/S3), model initialization (singleton pattern), and output generation (client-side rendering). It handles 109 languages and includes specialized modules for mathematical formula recognition (UniMERNet), table structure recovery (SLANet+/UNet), and office document normalization. While the code is actively maintained and functionally rich, it carries some technical debt from its PaddleOCR heritage (vendored code, eval-based builders) and has a complex optional dependency matrix for hardware accelerators.
AudioX-Turbo is a unified framework for efficient anything-to-audio generation (text, video, audio conditions β audio) using Distribution Matching Distillation (DMD) adapted to flow matching and adversarial training. It supports six core tasks: Text-to-Audio, Text-to-Music, Video-to-Audio, Video-to-Music, Text+Video-to-Audio, and Text+Video-to-Music. The system includes training scripts, an interactive Gradio demo, and a modular architecture with interchangeable components (attention, bottlenecks, etc.) via a factory pattern. Key innovations enable high-quality 4-step audio generation, reducing computational cost by up to 25Γ compared to multi-step diffusion models.
TinyLLaVA Factory is a modular Python framework for building, training, evaluating, and serving small-scale Large Multimodal Models (LMMs). It provides a pluggable architecture with separate components for vision towers (CLIP, SigLIP, DINOv2), language models (Phi, TinyLlama, Gemma, Qwen2, etc.), and connectors (MLP, QFormer, Resampler), all accessible via factory registries. The training pipeline supports two-stage pretraining (alignment) and finetuning with full, LoRA, or QLoRA recipes using DeepSpeed. It includes a comprehensive evaluation suite for standard benchmarks (VQA, GQA, SQA, POPE, MMMU, etc.) and a Gradio-based web demo. The codebase is well-organized and follows clean architectural patterns, though many modules are tightly coupled to the internal framework. It is actively maintained, with recent additions like a visualization tool and support for new models.
VieNeu-TTS is an advanced on-device Vietnamese text-to-speech library with instant voice cloning, supporting bilingual (Vietnamese/English) synthesis. The system provides a Python SDK (`vieneu`) with multiple backends: v3 Turbo (default, 48kHz, ONNX or PyTorch), v3 Nano (lightweight 24kHz, ONNX only), and legacy v1/v2 (deprecated). It includes a Gradio web UI (`vieneu-web`), a FastAPI streaming server, and a browser-based HTML/JS client. The core architecture is a factory pattern (`Vieneu(mode=...)`) that returns the appropriate engine, all inheriting from a shared base class. Inference can run entirely torch-free on CPU via ONNX Runtime, with optional GPU acceleration (CUDA) for batched generation and lower latency. Feature highlights include instant voice cloning from a 3β8 second reference clip using a speaker encoder (ONNX) and denoiser, inline emotion cues (`[cΖ°α»i]`, `[thα» dΓ i]`), frame-level streaming, SRT subtitle-to-speech, and LoRA fine-tuning for custom voices (requires ~6GB GPU). The system is deployed via Docker with CPU and GPU profiles, and has CI/CD pipelines for testing and PyPI publishing. The codebase is well-structured with clear separation of concerns, though it carries some legacy weight from older model versions and a few deprecated parameters. Other notable components include a sliding-window repetition penalty, a babble guard heuristic for short utterances, and a voice persistence system for user-defined presets.
Irodori-TTS is a state-of-the-art Japanese text-to-speech system built on rectified flow diffusion over DACVAE audio latents. It provides a complete training and inference stack: `train.py` supports full fine-tuning, LoRA, speaker inversion, and MeanFlow distillation with distributed training and WSD scheduling; `infer.py` offers a rich CLI with CFG, Sway Sampling, duration scaling, and watermarking; `gradio_app.py` and `gradio_app_voicedesign.py` expose interactive UIs for reference-based cloning and caption-conditioned voice design. The architecture is modular (`irodori_tts` package) with attention dispatch (FA3/SDPA), a hybrid Muon+AdamW optimizer, codec abstraction, and caching-friendly `InferenceRuntime`. While highly engineered for production use (INT4/INT8 quantization, safetensors, HuggingFace Hub integration), it remains a research-oriented codebase with experimental sampling algorithms and many YAML configs for different model sizes/versions. The code is clean and well-factored, though `train.py` is a large monolith and external dependencies (DACVAE, silentcipher) are pinned to specific commits, creating ecosystem fragility.
ChatTTS is a generative speech model application designed specifically for dialogue scenarios, functioning as a Text-to-Speech (TTS) library with integrated serving infrastructure. The core logic resides in the ChatTTS package, where `ChatTTS/core.py` defines the main inference interface. The architecture combines a GPT-based language model (`ChatTTS/model/gpt.py`) for token generation, a DVAE (`ChatTTS/model/dvae.py`) for audio compression, and Vocos as a vocoder. The system supports multiple speakers, prosodic control (laughter, pauses), and text normalization for Chinese and English. Deployment is handled through multiple entry points: a FastAPI-based API server (`examples/api/main.py`), a Gradio WebUI (`examples/web/webui.py`), and CLI tools (`examples/cmd/run.py`). A significant engineering highlight is the integration of vLLM (`ChatTTS/model/velocity/`) to optimize inference throughput for the GPT component. The repository includes robust asset management (`ChatTTS/utils/dl.py`) for model weights with SHA256 verification and ONNX export capabilities (`examples/onnx/exporter.py`) for cross-platform deployment. While the underlying model is research-grade (trained on 100k+ hours of audio), the repository structure prioritizes application delivery over experimental research. It provides a stable library interface with clear separation between model logic, audio processing (`tools/audio/`), and serving layers. The codebase demonstrates high craftsmanship in handling stochastic generation reproducibility (`tools/seeder/ctx.py`) and speaker embedding management (`ChatTTS/model/speaker.py`), making it a production-ready TTS solution rather than a raw research notebook collection.
Kotaemon is a production-grade Retrieval-Augmented Generation (RAG) application for chatting with documents. It provides a full-stack Gradio web UI with user management, document indexing (PDF, DOCX, HTML, images, web URLs), hybrid retrieval (vector + full-text), reranking, and multi-step reasoning pipelines (simple QA, question decomposition, ReWOO agent). The architecture is a monolith split into two Python libraries: `kotaemon` (core components: embeddings, LLMs, indices, storages, loaders, agents) and `ktem` (application layer: UI pages, reasoning pipelines, database models, settings). It supports multiple LLM providers (OpenAI, Azure, Anthropic, Gemini, Cohere, Ollama, LlamaCpp), embedding models, vector stores (Chroma, LanceDB, Milvus, Qdrant), and document stores. The system is highly configurable via environment variables and a settings UI. While primarily an application, it includes data/research elements such as document loaders, OCR pipelines, graph-based indexing (GraphRAG, LightRAG, NanoGraphRAG), and a training dataset for question rephrasing.
Stable Diffusion Web UI is a Gradio-based web application that provides a comprehensive interface for generating and manipulating images using diffusion models. It supports txt2img, img2img, inpainting, outpainting, and various post-processing operations including face restoration and upscaling. The application implements a plugin architecture through extensions, supports multiple model architectures (SD 1.x, SD 2.x, SDXL, SD3), and provides a REST API for programmatic access. The codebase is a monolithic application with tightly coupled modules handling model loading, image processing, UI rendering, and script execution. The architecture centers around a main webui.py entry point that initializes the application, sets up Gradio interfaces, and manages the event loop. The modules directory contains core functionality including sd_models.py for model management, processing.py for the main generation pipeline, and various hijack modules that monkey-patch diffusion model components to add custom behavior like attention optimization and CLIP modification. The extensions-builtin directory contains first-party extensions for LoRA, hypernetworks, and various post-processing operations. Key technical patterns include extensive use of monkey-patching and function hijacking to modify model behavior without changing upstream code, a callback system for script hooks, and a queue-based generation system with progress tracking. The codebase demonstrates sophisticated handling of GPU memory management, model optimization techniques, and cross-platform compatibility. However, the code exhibits significant technical debt with large monolithic files, tight coupling between modules, and complex conditional logic handling multiple model versions and configurations. The repository includes comprehensive test coverage for API endpoints, model utilities, and image processing functions. Dependencies are well-pinned in requirements_versions.txt, though some versions are notably behind current releases (Gradio 3.41.2, Transformers 4.30.2). The codebase shows active maintenance with recent commits and a detailed changelog tracking features and bug fixes across multiple versions.
VoxCPM is a tokenizer-free, multilingual text-to-speech (TTS) system developed by OpenBMB. It supports voice cloning, voice design, and controllable speech generation across 30 languages and 9 Chinese dialects, outputting 48kHz audio. The architecture is a diffusion-autoregressive model built on MiniCPM-4, comprising AudioVAE, Local Encoder, Text-Semantic LM, Residual Acoustic LM, and Local DiT. The repository provides multiple interfaces: a Gradio web application (app.py) for interactive use, a command-line interface (voxcpm.cli) for batch processing, a fine-tuning web UI (lora_ft_webui.py) for LoRA and full-parameter training, and Python APIs for integration. It also includes utilities for audio enhancement (ZipEnhancer), timestamp alignment, and text normalization. While primarily an application, it contains research-grade model definitions and training pipelines, making it a hybrid project with a strong application focus.
FastRTC is a Python library that enables developers to build real-time audio/video streaming applications with minimal code. It provides a high-level `Stream` class that integrates with Gradio for automatic UI generation and supports both WebRTC and WebSocket protocols. The library includes built-in handlers for voice activity detection (VAD) via Silero, speech-to-text via Moonshine, text-to-speech via Kokoro/Cartesia, and turn-taking logic. It also offers TURN credential helpers for Cloudflare, Twilio, and Hugging Face. The architecture revolves around defining a handler function that processes media chunks, wrapping it with turn-taking decorators, and creating a Stream object that handles signaling, connection management, and UI rendering. The codebase is actively maintained with frequent commits and an extensive set of demos integrating with various LLMs and AI services.
This repository implements the INP-Former model for anomaly detection in industrial inspection tasks. The system extracts Intrinsic Normal Prototypes (INPs) directly from test images using a DINOv2 vision transformer encoder, followed by an INP extractor and decoder to reconstruct normal regions. Reconstruction errors serve as anomaly scores. The repository provides training scripts for various settings (single-class, multi-class, few-shot, zero-shot, super-multi-class), a Gradio-based zero-shot application for interactive inference, ONNX conversion and inference tools, and a competition submission kit. It achieves state-of-the-art performance on standard benchmarks like MVTec-AD, VisA, and Real-IAD. The architecture is modular, separating the core model, dataset handling, utilities, and training logic, though training scripts show some duplication across different settings.
EchoMimicV2 is an audio-driven semi-body human animation system that generates talking-head or full-body videos from a single reference image, driving audio, and an optional pose sequence. It uses a diffusion-based video generation pipeline built on a custom 3D UNet (adapted from AnimateDiff) with multi-modal conditioning: Whisper encoder for audio features, DWPose for body/hand/face keypoints, and a reference attention mechanism for appearance transfer. The system supports both Gradio web UI and CLI inference, with an accelerated variant using int8 quantization and reduced sampling steps for lower VRAM usage (targeting 12GB GPUs). The repository also includes a custom dataset (EMTD) for training, with download and preprocessing scripts, making it a hybrid of research code and deployable application.
Bert-VITS2 is a multilingual text-to-speech system based on VITS2 (Variational Inference Text-to-Speech with improved Transformer backbone). It supports Chinese, Japanese, and English synthesis with optional emotion conditioning via pre-trained models (WavLM, wav2vec2-emotion, CLAP). The codebase provides a complete pipeline: data preprocessing (resampling, text normalization, BERT feature extraction), multi-speaker training with distributed support, inference through a Gradio web UI or FastAPI server (hiyoriUI.py), and ONNX export for deployment. The architecture uses separate per-language BERT models for text encoding, a flow-based decoder, stochastic duration predictor, and multi-scale discriminators (including a WavLM-based discriminator). The repository includes numerous versioned subdirectories (oldVersion/, onnx_modules/) for backward compatibility with model versions 1.0.1 through 2.4, resulting in significant code duplication and maintenance overhead. The README explicitly states the project is no longer maintained and recommends the successor project Fish-Speech.