$ show EternalEvan/Astra
The repository provides a comprehensive framework for generative AI models, particularly focused on image and video generation. It integrates various diffusion-based architectures (Stable Diffusion, Stable Diffusion XL, Hunyuan DiT, Wan, etc.) and offers utilities for model loading, prompt engineering, and post-processing. The system supports text-to-image, text-to-video, image-to-video, and video-to-video generation, along with editing capabilities via ControlNet and LoRA. The architecture is modular, with core components for model management, prompting, and VRAM optimization, though it exhibits some duplication across model-specific pipelines.
The LLM identified 5 files worth highlighting for reuse:
Centralized model loading and management system that handles weights from various sources (Hugging Face, Civitai, local), device placement, and dtype conversion. Demonstrates a clean abstraction for managing heterogeneous models in a generative AI pipeline.
Abstract base class for text encoding pipelines, with specific implementations for different models (e.g., WanPrompter, FluxPrompter). Prompt engineering logic is decoupled from model inference, allowing for reusable text conditioning across various generative models.
Provides AutoWrappedModule for selective CPU offloading of model components and dtype conversion to manage GPU memory. Enables efficient inference of large models by dynamically moving parts of the model to CPU as needed.
Allows chaining multiple image/video processors (e.g., for applying contrast, sharpness, and temporal smoothing in sequence). Implements a clean processor interface that promotes modularity and reuse of processing steps.
Defines a unit for a single ControlNet condition, with managers for combining multiple ControlNets. Provides a flexible abstraction for integrating conditional guidance into diffusion models, supporting various control signals like edges, depth, and pose.