This repository is a production-grade face manipulation platform (facefusion). It operates as a modular monolith, combining a powerful CLI, a Gradio web UI, a job queue system, and a plugin-based processor architecture. The system ingests images or videos, detects faces using various ONNX models (RetinaFace, YOLO, etc.), and applies a configurable pipeline of processors (face swapping, enhancement, editing, age modification, lip-syncing, background removal, etc.). It relies heavily on ffmpeg for video/audio handling and ONNX Runtime for model inference. The architecture is exceptionally clean, with clear separation of concerns between the UI, CLI, workflows, jobs, and individual processors. It is not a research notebook or a simple script; it is a well-engineered application designed for end-to-end media processing, demonstrating high craftsmanship in state management, thread safety, and extensibility.
LiveTalking is a real-time interactive streaming digital human engine that synchronizes lip movements with audio generated from text or speech input. It functions as a production-grade web server (Flask + aiohttp) supporting WebRTC, RTMP, and virtual camera output, enabling applications like live streaming, virtual assistants, and conversational avatars. The system integrates a pluggable pipeline: user input → optional LLM (OpenAI-compatible) → TTS (8+ providers including EdgeTTS, CosyVoice, GPT-SoVITS) → lip-sync model inference (Wav2Lip, MuseTalk, Ultralight Digital Human) → video frame compositing → streaming output. It also includes ASR via a local FunASR/SenseVoice WebSocket server and a web admin UI for session management and avatar generation. Architecturally, it follows a plugin-based design using a decorator registry (registry.py) that allows extending TTS, Avatar, and Output modules via registration. The core rendering loop in BaseAvatar manages three concurrent threads (TTS, inference, frame processing) with queue-based data flow. WebRTC integration uses aiortc for peer connections, and the system supports multi-session concurrency with session limits. While primarily an application, it embeds research-derived model code (Whisper, face detection, face parsing) and includes a benchmark script for ASR latency comparison. The codebase has some technical debt: duplicated face detection utilities, stale Dockerfile (Python 3.10/torch 1.12 vs README's newer requirements), and commented-out legacy code. However, the core architecture is solid, and the plugin system and audio utilities are well-crafted and reusable.