This is a large polyglot monorepo for WiFi-based human sensing (WiFi-DensePose) that turns commodity WiFi routers into spatial intelligence systems. It captures Channel State Information (CSI) from ESP32 nodes to detect presence, estimate 17-keypoint body pose, measure breathing and heart rate, and track movement through walls without cameras or wearables. The system integrates deeply with Home Assistant via MQTT and Matter, provides a HomeKit bridge, and exposes multiple surfaces: a Rust backend workspace (v2/crates) with 50+ crates for signal processing, privacy shielding (VEIL/BFLD), pose physics, and home automation; Python bindings (PyO3) for ML training and inference; TypeScript CLI and MCP tools for AI agent integration; ESP32-C6/S3 firmware with adaptive sensing; React Native mobile app; Lit-based dashboard; and Tauri desktop app. The architecture is event-driven with a streaming engine, coherence-gated privacy layers, and deterministic verification via SHA-256 golden files. The codebase is heavily ADR-driven with extensive test coverage including cross-language parity gates. While primarily an application, it also contains substantial research code (benchmarks, physics simulations, SOTA surveys) and is best described as a research-informed production platform.
Ultralytics is a production-grade computer vision library built around the YOLO model family. In production it provides a unified Python/CLI API for loading pretrained and custom models, running inference, training, validation, export, benchmarking, and object tracking across detection, instance and semantic segmentation, pose estimation, oriented bounding boxes, classification, and depth estimation. The architecture is a modular Python package: an engine layer (Model, BasePredictor, BaseTrainer, BaseValidator), a unified AutoBackend inference dispatcher supporting 20+ formats from PyTorch to TensorRT/ONNX/OpenVINO/CoreML/TFLite, a YAML-driven model configuration system, dataset loaders with caching and augmentation, pluggable experiment-tracking callbacks, and a family of ready-to-use solution classes for counting, heatmaps, queue management, and similar vision applications. The CLI entrypoint (`yolo`) and Python classes (`YOLO`, `SAM`, `RTDETR`, etc.) are the primary entry points, backed by active CI, Docker images, and extensive examples in Python, C++, and Rust.
X-AnyLabeling is a cross-platform desktop application (PyQt6) for AI-assisted annotation of images, videos, and 3D point clouds. It serves as a unified interface for a vast model zoo including YOLO (v5-v26), SAM (1/2/3), Grounding DINO, Florence-2, PP-OCR, and many others, enabling auto-labeling for detection, segmentation, pose estimation, OCR, tracking, and more. The application features a plugin-based model architecture, abstracted inference engines (ONNX, TensorRT, PyTorch), and a comprehensive settings system. It also integrates an Ultralytics YOLO training pipeline and a secure in-app update mechanism. While primarily an application, it has a strong hybrid nature due to its deep integration of research code (vendored Grounding DINO, SAHI) and data pipeline utilities (format converters, COCO evaluators). The codebase demonstrates high craftsmanship in its model abstraction layer and inference engine design, though the sheer volume of integrated models introduces a medium risk of technical drift. It is actively maintained with a large community and extensive test coverage.
NoPoSplat is a research application for novel view synthesis and camera pose estimation from unposed image pairs. It predicts 3D Gaussian parameters (means, covariances, harmonics, opacities) in canonical space using a vision transformer backbone (CroCo, ResNet, or DINO) with a DPT head, then renders images via differentiable Gaussian splatting. The system eliminates the need for known camera poses during training/inference by estimating poses implicitly through geometric consistency losses. Built on PyTorch Lightning with Hydra configuration, it supports multi-GPU training and evaluation on datasets like RealEstate10K, ACID, DTU, and ScanNet++. The codebase emphasizes modularity through component registries, composable loss functions, and chainable data preprocessing shims.
LightGlue is a PyTorch library for local feature matching between image pairs. It implements a deep neural network that takes sparse keypoints and descriptors (extracted by models like SuperPoint, DISK, ALIKED, or SIFT) and outputs correspondences. The architecture uses stacked self- and cross-attention transformer layers with adaptive pruning (width and depth) for speed, achieving state-of-the-art accuracy at high throughput. The repository provides pretrained weights, a benchmark script, and a demo notebook. It is designed for inference only; training code is in a separate repository (glue-factory). The code is well-structured, with clean separation between feature extractors, the matcher, and utility functions. It supports FlashAttention and PyTorch compilation for performance.
opentrack is a mature desktop head-tracking application (C++/Qt) that computes 6-DOF head pose from many input sources — IR point trackers, ArUco/paper markers, ONNX neural-network face pose estimation, IMU/serial devices, Wiimote, joysticks, eye trackers, and XR glasses — and streams it to games and flight simulators through output protocols such as Freetrack shared memory, TrackIR/NPClient emulation, SimConnect, UDP, X-Plane, virtual joystick (vJoy/libevdev), and a Wine/Proton bridge. The core is a plugin architecture defined in api/plugin-api.hpp (ITracker, IFilter, IProtocol) with dynamic loading, a processing pipeline (logic/pipeline.cpp) that applies axis mapping, centering, relative translation, and filtering, and a main Qt GUI (opentrack/main-window.cpp) for module selection and profile management. The application is organized as dozens of CMake-built plugin modules (tracker-*, proto-*, filter-*) around a compact core of shared infrastructure: a type-safe options/settings framework with versioned migrations, a spline-based response-curve system, cross-platform compatibility helpers (shared memory, timers, math, camera enumeration), and video abstraction. Notable strengths are the clean separation between modules, self-registering plugin macros, and genuinely reusable subsystems (spline library, UKF sigma-point header, settings framework, migration framework). Some modules are research-grade: the Alpha Spectrum adaptive filter (EMA/Brownian/predictive mixture with Rényi entropy), UKF-based deadzone filtering, and ONNX head-pose models. Weaknesses: the codebase carries significant legacy and platform fragmentation — dozens of optional SDK bindings (SimConnect, RealSense, Kinect, Tobii, SteamVR, Oculus Rift) that silently skip on missing SDKs, macOS is explicitly unmaintained, translations are largely unfinished stubs, several modules are experimental (tracker-papertracker, filter-alpha-spectrum), and some bundled third-party code (WiiYourself) has non-free terms. There are also signs of Qt6 migration in progress and mixed OpenCV version requirements. Overall this is an active, real-world, well-architected application with strong reusable components, though the long tail of legacy modules should not be mistaken for production quality.
This repository implements VPoser, a variational autoencoder-based human pose prior for SMPL/X body models. It provides tools to encode/decode human poses into a low-dimensional latent space, sample novel poses, and perform inverse kinematics fitting to 2D/3D key points using the learned prior. The codebase includes data preparation scripts for the AMASS dataset, a PyTorch Lightning trainer, a flexible IK engine with LBFGS/Adam optimizers, and utilities for rotation mathematics and linear blend skinning. The system is designed as a reusable library for pose prior applications in research and graphics. Architecture-wise, the project is organized into modules: body_model (SMPL/X loading and LBS), models (VPoser VAE and IK engine), tools (rotation conversions, 6D representation, file/path helpers), data (dataset loading and preparation), train (training scripts), and tutorials (Jupyter notebooks and example scripts). Entry points include training scripts (e.g., train/V02_05/V02_05.py), IK demonstrations (tutorials/ik_example_*.py), and usage tutorials (tutorials/vposer.ipynb). External dependencies are primarily PyTorch, Lightning, OmegaConf, and optional visualization packages. Data flows: For training, AMASS .npz files are converted to .pt tensors via prepare_data.py, loaded by VPoserDS, and fed into VPoserTrainer. For inference, pretrained models are loaded via model_loader.py, enabling pose encoding/decoding or IK fitting. The IK engine iteratively optimizes pose, shape, and translation in the VPoser latent space to match target key points.
Scal3R is a research codebase for CVPR 2026 Highlight paper on large-scale 3D reconstruction from unordered images using test-time training. The core contribution is a novel bidirectional attention mechanism with fast weight updates (TTT layers) that adapts during inference to improve reconstruction quality for large scenes. The repository provides an end-to-end inference pipeline: given a folder of images, it estimates camera poses, depth maps, and produces a colored point cloud. The architecture builds on a DINOv2 backbone and VGGT-based aggregator with custom attention heads, loop detection via SALAD, and pose graph optimization with submap management. The codebase is well-structured with separate modules for data loading, model definition, inference orchestration, and visualization tools. It includes fallback implementations for key components (e.g., Python-only loop solver, simple correlation-based loop detector) to improve portability, though GPU and CUDA are assumed. The repository is primarily a research artifact with clean utilities that could be extracted for other 3D vision projects. Evaluation code and dataset scripts are explicitly marked as not yet released, limiting reproducibility until those are provided.
CARI4D is the official CVPR 2026 research codebase for category-agnostic 4D reconstruction of human-object interaction from monocular video. It is fundamentally a research pipeline rather than a production application: raw RGB-D video (primarily BEHAVE, plus HODome/InterCap and custom clips) is processed through a chain of external foundation models - UniDepth for metric depth, NLF/SAM3D for human pose, FoundationPose for object tracking - then aligned, rendered into H5 training files, refined by a learned temporal transformer (CoCoNet), and finally optimized at test time to output temporally consistent SMPL-H body parameters, object 6D pose, and hand contact labels. The repo contains training, inference, preprocessing, and evaluation entry points, but no API/service layer; the runnable scripts are research pipeline drivers.
This repository provides a functional system for motion retargeting from human video to robot models. It detects arm and hand poses from monocular video using MediaPipe, fuses the detections, applies multi-stage coordinate transformations (world → first-person → URDF standard), and retargets the motion to robot URDF models using numerical optimization. The system includes visualization tools for debugging coordinate transformations and verifying retargeting results. The architecture is modular with separate components for detection (`src/whobody_dect/`), retargeting (`src/dex_retargeting/`), and example workflows (`example/`). Key entry points include `example/vector_retargeting/retarget_from_wholebody_video.py` for the main retargeting pipeline and `src/whobody_dect/simple_visualize.py` for coordinate system validation.
This repository is a web component library focused on accessibility, providing custom HTML elements for building inclusive web applications. It implements alternative input methods including standard mouse/keyboard/touch and pose/body tracking using TensorFlow's PoseNet model. The architecture follows web component principles using LitElement and Polymer, with a clear hierarchy of base classes (AbstractInputElement, AbstractUIElement) and specialized components for various UI patterns. The library emphasizes accessibility features like ARIA support, keyboard navigation, and screen reader compatibility. The codebase demonstrates strong craftsmanship in its utility implementations, particularly in coordinate transformation handling (mouse-position.ts), vector mathematics (vec2.ts), and general-purpose helper functions (utils.ts). The webcam integration (webcam.ts) shows proper resource management and lifecycle handling. However, the library suffers from significant technical drift due to outdated dependencies (lit-element 0.5.2, tfjs 0.13.0, typescript 3.1.6), which would require substantial updates for modern use. While the core architectural patterns are sound, the dependency versions limit practical reusability without migration work.