Helios is a real-time long video generation system that implements text-to-video (T2V), image-to-video (I2V), and video-to-video (V2V) generation using a 3D transformer architecture. The system features a modular design with specialized optimizations for performance including Ahead-of-Time Compilation (AOTI), group offloading for VRAM reduction, context parallelism (Ulysses/Ring attention), and Flash Attention kernel optimizations. It employs a multi-stage training pipeline (Base → Mid → Distilled) to balance quality and sampling speed. The repository includes a Gradio web interface (app.py) for interactive generation, command-line inference scripts, training utilities, and a comprehensive evaluation suite measuring video quality across aesthetic, motion, semantic, and naturalness dimensions. While containing research elements for ablation studies and benchmarking, its primary function is as a deployable video generation application.
VideoCoF is a unified video editing framework based on diffusion transformers (Wan2.1). It implements a 'See → Reason → Edit' pipeline using chain-of-frames (CoT) reasoning, enabling object removal, addition, swap, and local style transfer. The architecture includes a modular pipeline (`WanPipeline`) with flow matching solvers, VAE, text encoder, and transformer backbone. Inference can be done via a fast inference script (`fast_infer.py`) using 4-step DMD, batch inference via JSON, or through a Gradio UI. Training supports LoRA fine-tuning with DeepSpeed ZeRO-2/3 for 1.3B and 14B models, with custom dataset classes (including CoT and joint image-video). External dependencies include Diffusers, Transformers, Gradio, OpenCV, DeepSpeed, and optional paifuser for optimizations. While primarily an application for video editing, the repo has strong research character with extensive evaluation metrics (CLIP, GPT), configuration files, and experiment scripts. The code is modular in parts but contains experimental features, hardcoded paths, and reliance on an internal optimization package (paifuser) that may limit external reuse. The core utility modules (LoRA, caching, attention, schedulers) are well-crafted and potentially reusable across other diffusion model projects.