$ show VAST-AI-Research/SkinTokens
SkinTokens is a research codebase for a novel 3D character rigging method (TokenRig) that learns a compact discrete representation (SkinTokens) for skinning weights. The system ingests 3D meshes with skeleton and skinning data (from ArticulationXL, VRoid Hub, ModelsResource), transforms them through a pipeline of augmentation, order arrangement, vertex group extraction, and sampling, then trains an FSQ-CVAE to compress skinning weights into discrete tokens. A Qwen3-0.6B-based autoregressive transformer (TokenRig) generates the full rig (skeleton hierarchy + skin tokens) as a single token sequence. The repo includes a Blender-based server for mesh loading and export, a Gradio demo, a CLI, and extensive data processing modules. The code is well-structured with dataclass-based configs, abstract interfaces, and modular design, but remains primarily a research artifact with some incomplete training implementations (e.g., TokenRig training_step raises NotImplementedError). Its hybrid nature leans strongly toward data research, as the core contribution is a learned representation and generative model for animation rigging.
The LLM identified 4 files worth highlighting for reuse:
The `voxel_skin` function propagates skinning weights from mesh vertices to a voxel grid using a graph-based shortest-path approach. It constructs a combined graph of mesh vertices, faces, grid points, and bone joints, then computes geodesic distances to assign weights. This is a reusable, self-contained algorithm for any mesh-to-voxel skinning transfer, demonstrated by its use in post-processing and vertex group extraction.
Provides a comprehensive set of augmentation classes for 3D character data, including random joint deletion, subtree dropping, skeleton collapse, linear/affine transformations, LBS pose perturbation, joint discretization, and vertex/normal jitter. Each augment is a dataclass with a `transform(asset)` method, making them composable and configurable via YAML/JSON. This pattern is reusable for any 3D mesh+skeleton data pipeline.
Implements a tokenizer for skeleton hierarchies that encodes joint positions as discrete coordinates, uses branch tokens for non-sequential joints, and inserts part tokens (e.g., 'body', 'hand') to denote anatomical regions. The tokenizer supports multiple skeleton templates (Mixamo, VRoid) and produces a reversible token sequence. It is a self-contained module that can be adapted to other skeleton representation tasks.
Implements Finite Scalar Quantization (FSQ), a learned discrete representation method that quantizes latent vectors to a fixed codebook via rounding in a bounded space. It supports multi-codebooks, projection layers, and straight-through gradient estimation. The module is extracted from the original FSQ paper and is reusable for any discrete representation learning task (e.g., VQ-VAE alternatives).