This repository implements a complete reinforcement learning pipeline for whole-body motion tracking on humanoid robots, specifically Unitree G1 and XBot L7. The system uses a teacher-student architecture with Proximal Policy Optimization (PPO) to train policies that track reference motions from retargeted motion capture datasets. It features three training phases: a base training phase, an adaptation phase with domain randomization, and a finetuning phase. The environment is built on MJLab (a MuJoCo wrapper) and includes extensive domain randomization (e.g., motor parameters, body mass, center of mass, joint offsets, external forces, and a window load capacity curriculum for payload-aware training). The codebase is highly modular, with registered MDP components (observations, rewards, terminations, randomizations) and a flexible configuration system using Hydra. It also includes advanced features such as symmetry augmentation, differentiable forward kinematics, and a custom motion dataset pipeline with memory-mapped storage. While primarily a reinforcement learning training framework, it has strong data engineering aspects for processing motion datasets and generating training data for the student encoder. The repository is well-structured for research and sim-to-real transfer, but it is tightly coupled to specific robot assets and dependencies (torch, torchrl, mjlab) with pinned versions that may require adaptation for other environments.
This repository provides an end-to-end reinforcement learning framework for Unitree legged robots (Go2, G1, H1, H2, A2, AS2, R1). It uses MuJoCo physics (via the custom `mjlab` package) to simulate locomotion and motion imitation tasks. The codebase includes training scripts (`scripts/train.py`) that run PPO (from `rsl_rl`) with configurable network architectures and multi-GPU support, exporting trained policies to ONNX format. The `deploy/` directory contains C++ applications that execute these policies on real hardware using ONNX Runtime and Unitree SDK2, orchestrated by a finite state machine. The system is split into two main tasks: velocity tracking (gait generation) and motion tracking (whole-body imitation from reference trajectories). Key architectural patterns include manager-based environment configuration, composable MDP terms (rewards, observations, terminations), curriculum learning, and a YAML-driven FSM for deployment. The repository is actively maintained, with recent commits and bundled third-party dependencies (MuJoCo binaries, ONNX Runtime) for cross-platform deployment.
SimScale is a research codebase for data-driven, non-reactive autonomous vehicle simulation and benchmarking, developed by OpenDriveLab. It provides a comprehensive pipeline for training and evaluating end-to-end driving policies on the nuPlan/OpenScene dataset, with support for synthetic data augmentation (SimScale) and multiple agent architectures including Transfuser, DiffusionDrive, and GTRS-Dense. The repository implements a Plan-Do-Measure (PDM) closed-loop scoring framework, metric caching, and a Hydra-based configuration system for flexible experiment management. While primarily a research platform, it includes production-like training scripts, evaluation pipelines, and submission tools for the NAVSIM v2 challenge. The codebase is heavily coupled to the nuPlan ecosystem and requires large-scale data downloads (>1TB), but offers reusable components such as abstract agent interfaces, feature/target builders, and batch IDM policies. It is actively maintained with recent commits and a growing community of extensions.
This repository implements a hierarchical controller for quadruped locomotion that combines a high-level reinforcement learning policy with a low-level model predictive controller (MPC). The RL policy dynamically predicts weight parameters for the MPC, which computes optimal ground reaction forces and joint torques. The codebase includes Isaac Gym training environments for the Unitree A1, Go1, and Aliengo robots, along with a custom C++ MPC solver (OSQP-based) and a finite state machine for robust transient handling. While primarily a research project that bridges RL and MPC for legged robots, it also provides reusable robotics components such as state estimators, leg controllers, gait pattern generators, and foot swing trajectory planners. The work is based on the MIT Cheetah software and includes detailed development logs and configuration files for reproducible experiments.