$ show Netflix/void-model
VOID is a Netflix research codebase for removing objects from videos while also erasing their physical side effects, such as shadows, reflections, and secondary motion caused by the removed object. The core runtime is a two-pass diffusion inpainting system built on CogVideoX: pass 1 performs coarse object removal guided by a quadmask, and pass 2 refines temporal consistency using warped noise and optical-flow-inspired conditioning. The repo is primarily an application/research library rather than a pure data project: it ships a `videox_fun` Python package with model definitions, custom diffusion pipelines, inference and training scripts, a FastAPI/Ray inference API, Gradio UIs, and a Colab notebook, alongside synthetic data-generation code that uses Blender and Kubric.
The LLM identified 5 files worth highlighting for reuse:
Converts large diffusion model weights to float8 and wraps forward passes in autocast, enabling video models to run on significantly smaller GPUs. This is a self-contained optimization utility that can be extracted for deploying any large PyTorch transformer or diffusion model under memory constraints.
Implements a distributed discrete timestep sampler for diffusion training, coordinating timestep selection across ranks to improve noise schedule distribution. The pattern is reusable in any diffusion training loop that wants balanced, distributed timestep coverage.
A PyTorch sampler that buckets mixed image/video samples by aspect ratio and content type, reducing padding waste and improving batch efficiency during multi-resolution generative model training. It is a portable data-loading pattern for any vision or video training pipeline.
Wraps RAFT optical-flow extraction with resolution-aware normalization and converts flow fields into HSV-to-RGB motion encodings, following the VideoJAM-style motion representation. This utility is reusable for injecting motion priors into video diffusion models or any flow-based video task.
Implements TeaCache-style timestep-aware caching that reuses transformer features and skips redundant denoising computations, accelerating video diffusion inference with limited quality loss. The approach is modular enough to be adapted to compatible diffusion pipelines beyond this repository.