$ show Tencent-Hunyuan/HunyuanVideo-1.5
HunyuanVideo-1.5 is an open-source video generation model by Tencent, implemented as a Python library with CLI scripts for inference (`generate.py`) and training (`train.py`). It generates high-quality videos from text or image prompts using a diffusion transformer (DiT) architecture with 8.3B parameters, supporting resolutions up to 1080p via super-resolution. The codebase provides a modular pipeline (`HunyuanVideo_1_5_Pipeline`) that orchestrates prompt encoding (byT5 text encoder, SigLIP vision encoder), denoising with flow matching, and VAE decoding. It includes advanced features like SSTA (Selective and Sliding Tile Attention) for efficient long-video processing, step distillation, CFG distillation, and various caching mechanisms (DeepCache, TeaCache, TaylorCache). The repository also offers a training script with distributed support (FSDP, context parallelism), LoRA fine-tuning, and the Muon optimizer. While primarily an application for video generation, it contains research-grade model implementations and evaluation benchmarks. The code is well-structured with reusable neural network modules, but relies on many optional hardware-specific dependencies (flash-attn, flex-block-attn, SageAttention) and external services for prompt rewriting, which may introduce operational complexity. The license is a custom Tencent Hunyuan Community License with restrictions.
The LLM identified 5 files worth highlighting for reuse:
Implements SSTA (Selective and Sliding Tile Attention), a novel sparse attention mechanism for long video sequences. It provides STA (Sliding Tile Attention), MOBA (Mixed Overlap Block Attention), and SSTA masks. This module is self-contained, with configurable block sizes and overlap, and can be extracted for use in other transformer-based video or long-sequence models.
Implements the Muon optimizer, which applies Newton-Schulz orthogonalization to the gradient before the standard AdamW update. This is a standalone optimizer class that can be reused in any PyTorch training loop. It demonstrates non-obvious optimization technique and is well-documented with references.
A custom discrete flow matching scheduler for diffusion models, supporting linear interpolation schedules, flux shift, and various timestep sampling strategies. It is a self-contained scheduler class that extends diffusers' SchedulerMixin and can be reused in other flow matching pipelines.
Provides CacheHelper for implementing DeepCache, TeaCache, and TaylorCache strategies during inference. It wraps transformer blocks to cache intermediate features and skip redundant computations. This module is isolated and can be adapted to other transformer-based generative models for speed improvements.
Contains distributed communication utilities for sequence parallelism, including all_to_all_4D, all_gather, and sync_tensor_for_sp functions. These are low-level, reusable primitives for implementing custom parallel strategies in PyTorch distributed training, independent of the video generation domain.