Capybara is a unified visual creation model that provides inference capabilities for text-to-video, text-to-image, instruction-based video-to-video, and instruction-based image-to-image tasks. The system features a modular architecture with separate components for model handling, pipeline orchestration, and utility functions. Key entry points include the main inference script, ComfyUI custom nodes, and batch/single inference scripts. The system supports distributed multi-GPU inference, prompt rewriting using Qwen3-VL, and FP8 quantization for reduced memory usage.
This repository implements a two-stage generative image model: Representation Autoencoders (RAE) followed by Diffusion Transformers (DiT^DH). Stage 1 trains a ViT decoder to reconstruct images from frozen pretrained encoders (DINOv2, MAE, SigLIP), with optional GAN and LPIPS losses. Stage 2 trains a class-conditional diffusion transformer on the resulting latent space, supporting both classifier-free guidance and autoguidance. The codebase is config-driven (OmegaConf), PyTorch DDP, and provides training, distributed sampling, FID evaluation, and latent statistics computation. It is a research codebase with strong engineering quality, modular design (transport, discriminator, encoders, decoders), and reproducibility through pinned dependencies and checkpointing.
HunyuanVideo-1.5 is an open-source video generation model by Tencent, implemented as a Python library with CLI scripts for inference (`generate.py`) and training (`train.py`). It generates high-quality videos from text or image prompts using a diffusion transformer (DiT) architecture with 8.3B parameters, supporting resolutions up to 1080p via super-resolution. The codebase provides a modular pipeline (`HunyuanVideo_1_5_Pipeline`) that orchestrates prompt encoding (byT5 text encoder, SigLIP vision encoder), denoising with flow matching, and VAE decoding. It includes advanced features like SSTA (Selective and Sliding Tile Attention) for efficient long-video processing, step distillation, CFG distillation, and various caching mechanisms (DeepCache, TeaCache, TaylorCache). The repository also offers a training script with distributed support (FSDP, context parallelism), LoRA fine-tuning, and the Muon optimizer. While primarily an application for video generation, it contains research-grade model implementations and evaluation benchmarks. The code is well-structured with reusable neural network modules, but relies on many optional hardware-specific dependencies (flash-attn, flex-block-attn, SageAttention) and external services for prompt rewriting, which may introduce operational complexity. The license is a custom Tencent Hunyuan Community License with restrictions.
SkyReels-V3 is a multimodal video generation inference framework that supports three core tasks: reference-to-video (generating video from 1-4 reference images), video extension (single-shot and shot-switching), and talking avatar generation (from a portrait and audio). The codebase implements a diffusion transformer architecture (Wan-based) with custom components including a causal 3D VAE, T5 text encoder, CLIP image encoder, wav2vec2 audio encoder, and a flow-matching scheduler. It provides multi-GPU support via xDiT USP and low VRAM options with FP8 quantization and block offloading. The system is designed as a CLI tool with a single entry point (`generate_video.py`) that dispatches to task-specific pipelines. While primarily an application for running inference, it contains several reusable modules such as a custom scheduler, attention wrappers, and color transfer utilities.