$ show microsoft/TRELLIS.2
TRELLIS.2 is a research codebase for state-of-the-art image-to-3D generation using a novel sparse voxel representation (O-Voxel) and flow matching with 4B parameters. It provides full training and inference pipelines for generating high-fidelity 3D meshes with PBR materials from single images. The repository includes custom CUDA extensions for voxel processing, a flexible sparse tensor library, and comprehensive dataset tooling for large-scale 3D asset curation (Objaverse-XL, Sketchfab). While primarily a research project with training code and data pipelines, it also offers Gradio web demos and Python APIs for inference. The codebase demonstrates strong engineering in its modular design (pipelines, mixins, config-driven training) but suffers from some research-typical issues like code duplication, hardcoded paths, and fragile dependency management.
The LLM identified 4 files worth highlighting for reuse:
Python wrapper for the C++/CUDA Flexible Dual Grid conversion, which converts meshes to sparse voxel grids with dual vertices computed via QEF (Quadratic Error Function). This is a core reusable component for any 3D voxelization task, providing clean typed entry points (mesh_to_flexible_dual_grid, flexible_dual_grid_to_mesh) with configurable resolution and feature channels. The underlying C++ implementation handles triangle-voxel intersection, TBN computation, and mipmap texture sampling, making it suitable for high-quality PBR material transfer.
Defines SparseTensor and VarLenTensor abstractions for batched sparse data with variable-length layouts. Provides caching for spatial operations (down/upsampling, neighbor maps), broadcasting, reductions, and backend-agnostic support (torchsparse, spconv). This is a highly reusable pattern for any project dealing with sparse 3D data, decoupling the data structure from specific sparse convolution libraries.
Implements a hierarchy of flow matching samplers: FlowEulerSampler, FlowEulerCfgSampler (with classifier-free guidance), and FlowEulerGuidanceIntervalSampler (with guidance interval scheduling). These are clean, composable samplers that can be reused for any flow matching model, not just 3D generation. The mixin-based design allows easy extension.
Provides ElasticModule and ElasticModuleMixin for adaptive memory management during training, using a MemoryController to dynamically offload and recompute activations based on VRAM pressure. This is a reusable pattern for training large models on limited GPU memory, applicable to any PyTorch model.