EchoMimicV3 is a unified multi-modal and multi-task human animation model that generates realistic human videos from audio and/or text prompts. Built upon the Wan2.1-Fun-V1.1-1.3B-InP base model, it supports both talking-head and full-body animation with two versions: preview (standard quality) and flash (optimized for speed and lower VRAM). The system provides Gradio-based interactive demos (app.py, app_mm.py) for real-time video generation from user-uploaded media, command-line inference scripts (infer_flash.py, infer_preview.py, run_flash.sh) for batch processing, and deployment options for ComfyUI, ModelScope, and HuggingFace. Core functionality centers on audio-driven lip-sync and facial animation, with optional text guidance for pose/scene control, leveraging diffusion models, transformer architectures, and multimodal encoders (CLIP for images, T5 for text, Wav2Vec2 for audio).
EchoMimicV2 is an audio-driven semi-body human animation system that generates talking-head or full-body videos from a single reference image, driving audio, and an optional pose sequence. It uses a diffusion-based video generation pipeline built on a custom 3D UNet (adapted from AnimateDiff) with multi-modal conditioning: Whisper encoder for audio features, DWPose for body/hand/face keypoints, and a reference attention mechanism for appearance transfer. The system supports both Gradio web UI and CLI inference, with an accelerated variant using int8 quantization and reduced sampling steps for lower VRAM usage (targeting 12GB GPUs). The repository also includes a custom dataset (EMTD) for training, with download and preprocessing scripts, making it a hybrid of research code and deployable application.