This repository provides a distributed training framework for diffusion models (image and video generation). It supports an extensive range of state-of-the-art models (Flux, SDXL, HunyuanVideo, Wan, Cosmos, etc.) and fine-tuning methods (LoRA, full fine-tune) using DeepSpeed pipeline parallelism. The architecture is built around a `BasePipeline`/`ComfyPipeline` abstraction that decomposes each model into pipeline layers, enabling memory-saving techniques like block swapping (CPU offloading), latent/text embedding caching, and custom optimizers (Automagic, AdamW8bitKahan). The project is heavily integrated with ComfyUI for model definitions and checkpoint formats. While primarily a training application, it includes significant research-oriented components such as advanced optimizers, low-rank gradient projectors, and CFG-augmented training, giving it a hybrid application/research character. The codebase demonstrates high craftsmanship in its pipeline parallelism design and memory optimization, but is coupled to DeepSpeed, PEFT, and many external submodules, and exhibits some duplication across model implementations.