$ show showlab/Show-o
Show-o and Show-o2 are research repositories for unified multimodal understanding and generation. They implement transformer-based models (1.3B, 1.5B, 7B parameters) that can perform text-to-image generation, image/video captioning, visual question answering, inpainting/extrapolation, and mixed-modality generation (interleaved text and images). The codebase is organized around two major model versions: Show-o uses discrete tokens with MAGVITv2 VAE and Phi-1.5 LLM, while Show-o2 moves to continuous latent space with WanVAE (3D causal VAE) and Qwen2.5 LLM. Training follows a multi-stage pipeline (pretraining, instruction tuning) with novel techniques like flow matching (via a modular `transport` package), dual-path vision encoders (SigLIP + patch embedding), and omni-attention masks that combine causal and bidirectional attention for multimodal sequences. The repository includes distributed training configs (DeepSpeed, multi-node), evaluation scripts for GenEval and DPG-Bench, and comprehensive YAML configuration files. This is primarily a data_research codebase: it produces research artifacts (models, papers at ICLR 2025 and NeurIPS 2025) rather than a production application. The code exhibits good modularity for a research project, with reusable components like the transport module, omni-attention masks, mixed dataloader, and attention masking utilities. However, it suffers from typical research-code issues: hardcoded internal infrastructure paths, reliance on non-public data pipelines (e.g., internal packages for RefinedWeb), and some code duplication between Show-o and Show-o2. Reproducibility is moderate—dependencies are pinned and configs are detailed, but many datasets are large-scale web sources that require manual setup, and the training scripts assume specific hardware (multi-node A100 clusters). The project is actively maintained (commits in early 2026) with a roadmap for future work, including mixed-modal generation and video support.
The LLM identified 5 files worth highlighting for reuse:
Implements a standalone Transport class supporting multiple flow-matching paths (Linear, VP, GVP) and a Sampler with ODE/SDE/DPM solvers. The design is cleanly decoupled: path definition, probability flow ODE, and sampling can be reused independently for any diffusion or flow-matching project. Demonstrates strong software craftsmanship in mathematical modeling and numerical methods.
Provides functions to create attention masks that mix autoregressive (causal) and bidirectional (full) attention patterns over multimodal sequences (text and image tokens). Uses PyTorch's FlexAttention for efficient compilation. The masking logic is generic and can be extracted for any transformer that needs per-token attention control in multimodal generation/understanding.
Implements MixedDataLoader to combine multiple DataLoaders with weighted sampling strategies (max_size_cycle, min_size, concat_max_size_cycle). This is a common need in multi-task or multi-dataset training and is implemented as a clean, reusable wrapper. It handles shuffling, resampling, and yielding dicts with dataset names, making it easy to integrate into any training loop.
Contains UniversalPrompting class that formats multimodal prompts with special tokens (<|soi|>, <|eoi|>, <|mmu|>) and creates corresponding attention masks. The interface is self-contained and parameterized (e.g., conditional dropout). This logic is essential for any multimodal LLM that tokenizes images into visual tokens and can be reused with different tokenizers or special token schemes.
Provides an EMA (Exponential Moving Average) utility class with warmup, state dict injection, and checkpoint-friendly design. The implementation is generic and not tied to any specific model, making it a drop-in component for any PyTorch training script that requires EMA tracking for stability or evaluation.