The system is a comprehensive toolkit for AI-driven image and video generation, primarily leveraging diffusion models such as Stable Diffusion, Stable Diffusion XL, and Stable Video Diffusion. It provides both a command-line interface (CLI) and an HTTP API server for generating images from text prompts, performing image-to-image translation, inpainting, ControlNet-guided generation, upscaling, face restoration, and video frame interpolation. The architecture is modular, separating core model implementations, enhancements, processors, and utilities, with a strong emphasis on extensibility through configurable backends and pluggable components. Key features include support for multiple sampling algorithms (DDIM, PLMS, K-diffusion solvers), integration with ControlNet for conditional generation via edge, depth, pose, and other maps, and post-processing pipelines for upscaling (using RealESRGAN or similar) and face enhancement (using CodeFormer). The system also incorporates multimodal capabilities through integrated models like BLIP for image captioning and visual question answering, and segment anything model (SAM) for segmentation tasks. Model weight management is handled through a centralized system that resolves and converts weights from various sources (Hugging Face Hub, local cache) and formats (CompVis, Diffusers, Refiners). The CLI offers commands like `imagine` for text-to-image generation, `aimg` as a multi-command interface for imagine, upscale, edit, server, and videogen operations, and `run-server` to launch the HTTP API with a web interface (StableStudio). The HTTP API provides REST endpoints for programmatic access, complete with OpenAPI documentation and GPU-aware concurrency control to handle multiple requests safely. Overall, the system is designed to be user-friendly yet powerful, automating complex workflows like model downloads and caching while providing low-level control for advanced users.
This repository is a modified/forked version of the Hugging Face Diffusers library with specific additions for RealRestorer, a generalizable real-world image restoration model. It provides pipelines for various image restoration tasks (blur removal, compression artifact removal, lens flare removal, moire removal, dehazing, low-light enhancement, denoising, rain removal, reflection removal), tools for synthesizing real-world degradations, benchmark evaluation capabilities, and standard diffusion model functionalities. The architecture follows a modular library structure with lazy loading patterns, standardized pipeline interfaces, and extensive training/inference utilities. Key entry points include CLI commands for export, inference, and degradation synthesis, along with server examples for deployment.
This repository is a ComfyUI custom node pack that extends the standard ControlNet functionality with advanced scheduling, weighting, and masking capabilities. It provides nodes for timestep and latent keyframes, custom weight curves, attention masks, effect masks, and sliding context windows for video generation (AnimateDiff integration). The system supports multiple control types: ControlNet, T2IAdapter, ControlLoRA, ControlLLLite, SparseCtrl, SVD-ControlNet, Reference, ControlNet++, and CtrLoRA. It also replicates features from Auto1111 like 'My prompt is more important' via soft weights and uncond_multiplier. The architecture is modular, with separate files for each control family (control.py, control_reference.py, control_ctrlora.py, etc.), a shared utilities module (utils.py) providing base classes like AdvancedControlBase and ControlWeights, and a node registration layer (nodes.py, nodes_main.py, etc.). The system integrates deeply with ComfyUI's internal APIs (comfy.controlnet, comfy.model_patcher, comfy.hooks) and uses monkey-patching to inject advanced behavior into standard control objects. A custom inter-node communication protocol (DinkLink) enables coordination with AnimateDiff-Evolved. Key entry points are the node classes registered in __init__.py and nodes.py, which are loaded by ComfyUI's extension system. The primary data flow involves loading a control model, optionally converting it to an advanced version, applying timestep/latent keyframes and weights, and then applying the control during sampling with optional reference injection or sliding context. The system is actively maintained with recent commits, CI publishing, and example workflows demonstrating advanced features like Qwen Image Inpainting and Anima LLLite v2.
The repository provides a comprehensive framework for generative AI models, particularly focused on image and video generation. It integrates various diffusion-based architectures (Stable Diffusion, Stable Diffusion XL, Hunyuan DiT, Wan, etc.) and offers utilities for model loading, prompt engineering, and post-processing. The system supports text-to-image, text-to-video, image-to-video, and video-to-video generation, along with editing capabilities via ControlNet and LoRA. The architecture is modular, with core components for model management, prompting, and VRAM optimization, though it exhibits some duplication across model-specific pipelines.
RepLDM is a research codebase accompanying a NeurIPS 2025 paper that introduces a training-free method for generating high-resolution images (up to 8K) using pretrained latent diffusion models (SDXL). The core innovation is an Attention Guidance module that enhances detail and color richness by applying a lightweight self-attention transformation to latent representations during denoising. The code provides custom diffusers pipelines (RepLDMSDXLPipeline, RepLDMSDXLControlNetPipeline, FreeScaleSDXLPipeline) that implement a progressive upsampling strategy: initial denoising at base resolution, then iterative VAE encode/decode with additional denoising steps to increase resolution. It supports text-to-image generation and ControlNet-conditioned generation. The repository includes Jupyter notebooks for interactive use and a batch inference script. While primarily a research artifact, the Attention Guidance module is a clean, self-contained component that could be reused in other diffusion pipelines. The code is tightly coupled to specific versions of diffusers and PyTorch, and some internal APIs are used, indicating potential drift with newer library versions.