$ show NVlabs/CARI4D
CARI4D is the official CVPR 2026 research codebase for category-agnostic 4D reconstruction of human-object interaction from monocular video. It is fundamentally a research pipeline rather than a production application: raw RGB-D video (primarily BEHAVE, plus HODome/InterCap and custom clips) is processed through a chain of external foundation models - UniDepth for metric depth, NLF/SAM3D for human pose, FoundationPose for object tracking - then aligned, rendered into H5 training files, refined by a learned temporal transformer (CoCoNet), and finally optimized at test time to output temporally consistent SMPL-H body parameters, object 6D pose, and hand contact labels. The repo contains training, inference, preprocessing, and evaluation entry points, but no API/service layer; the runnable scripts are research pipeline drivers.
The LLM identified 5 files worth highlighting for reuse:
Standalone PyTorch implementation of SMPL/SMPL-H forward kinematics with gender-specific body models and hand articulation. It exposes a clean layer interface with no internal project imports, making it directly reusable in any PyTorch project needing SMPL body parameterization, priors, or landmark regression.
Provides conversions among axis-angle, quaternion, rotation matrix, and 6D rotation representations, plus geodesic distance and point-cloud transforms. Dependency-light, dataset-agnostic, and safely extractable for pose estimation, SLAM, calibration, or graphics code.
A composable set of RGB and depth augmentation primitives: brightness/color jitter, blur, JPEG compression, depth-correlated Gaussian noise, missing-depth dropout, ellipse dropout, and background replacement. Each transform is isolated and parameterized, so the module can be reused in other RGB-D training pipelines with minimal adaptation.
Implements frequency embeddings, residual MLP blocks, and adaptive layer-norm conditioning. This is a generic building block for injecting temporal or scalar conditioning into neural networks. It imports no internal project modules and can be lifted directly into a new model architecture.
Robust scale/shift estimation, bilateral filtering, and erosion utilities used to align predicted depth maps to metric depth. The operations are dataset-agnostic and applicable to any monocular depth estimation or fusion pipeline that needs metric alignment.