Alpamayo 1 is a Vision-Language-Action (VLA) model designed for autonomous driving applications. It implements the Alpamayo-R1 architecture that combines a Qwen3-VL backbone for vision-language reasoning with a diffusion-based action expert for trajectory prediction. The system takes multi-camera video and egomotion history as inputs, generates Chain-of-Causation (CoC) reasoning traces, and outputs future trajectories (6.4s horizon at 10Hz). Key components include a unicycle kinematic model for action space conversion, CUDA graph acceleration for efficient repeated inference, and robust geometric transformations. While the repository is noted to receive only limited maintenance as newer versions exist, the current implementation demonstrates strong engineering practices with several reusable components.
FluxVLA is a comprehensive, config-driven framework for training, evaluating, and deploying Vision-Language-Action (VLA) models for robotic manipulation. It supports multiple VLA architectures (OpenVLA, Pi0/Pi0.5, GR00T, Cosmos3, DreamZero, etc.) and robot platforms (LIBERO simulation, RoboCasa, Franka, UR3, ALOHA, Oli humanoid, Tron2). The codebase provides a modular pipeline with registries and builders for datasets, transforms, models (backbones, heads, projectors), optimizers, collators, and inference runners. Production capabilities include distributed training (FSDP/DDP), real-time ROS inference, custom CUDA/Triton kernels for performance, and evaluation with metrics reporting to Feishu. While primarily an engineering framework for applied robot learning, it also contains research contributions such as flow-matching action heads, reward models (ARM, SARM), and real-time chunking (RTC) guidance. The architecture is clean and extensible, though config proliferation and vendored third-party code introduce maintenance overhead.
The solo-cli is a command-line tool designed to deploy and manage Physical AI systems on local hardware. It integrates functionalities for serving large language models (via Ollama, vLLM, or llama.cpp), controlling robotic arms (through the LeRobot framework), authenticating with a custom model hub (Solo Hub), and fine-tuning models using external datasets. The application features a modular command structure with lazy-loaded dependencies to optimize startup time, cross-platform hardware detection for automated backend selection, and a layered configuration system. Additional utilities include Docker management, system information gathering, and filesystem-safe string handling for robotics data operations. The project also includes example implementations of Model Context Protocol (MCP) servers for various domains, demonstrating how to extend the CLI's capabilities for specialized AI workflows.
Evo-1 is a research codebase for a lightweight Vision-Language-Action (VLA) model accepted at CVPR 2026. It provides a complete pipeline for training (two-stage pretraining + finetuning using DeepSpeed and InternVL3-1B backbone) and evaluating robot manipulation policies on simulation benchmarks (Meta-World, LIBERO, LIBERO-Plus, RoboTwin) and real robots (xArm6, Aloha, SO100). The primary data flow involves converting raw robot datasets (LeRobot v2.1 format) into normalized state/action pairs with delta-timestep support, computing per-dataset norm statistics, and training a flow-matching action head with a frozen VLM backbone. The repo also includes a WebSocket-based inference server and clients for real-robot deployment. Although the core is research-oriented, it incorporates the full LeRobot framework for data collection and robot control, giving it a hybrid character. The codebase is well-structured with reusable components for dataset processing, normalization, and video handling, but the embedded LeRobot copy and multiple evaluation environments introduce maintenance overhead.
Visual Spatial Tuning (VST) is a research framework for enhancing Vision-Language Models (VLMs) with human-like visuospatial abilities, from perception to reasoning. It implements a two-stage pipeline: supervised fine-tuning (SFT) on 4.1M spatial perception samples (VST-P) followed by reinforcement learning (RL) on 135K reasoning samples (VST-R). The framework extends to Vision-Language-Action (VLA) models for robotic manipulation evaluated on the LIBERO benchmark. The repository contains custom model implementations (Qwen2.5VL, Qwen3VL) with sequence parallelism and FSDP support, data preprocessing scripts, evaluation benchmarks (VSIBench, 3D object detection on SUN RGB-D, VLM benchmarks), and a distributed RL training system (verl) with reward functions and advantage estimation. While primarily a research codebase, it includes production-quality training infrastructure and reusable data utilities. The code is actively maintained with recent commits, but has medium technical drift due to tight version pinning and reliance on an internal ByteDance framework (VeOmni) that is not publicly available, which may hinder full reproducibility.
Microsoft Magma is a research codebase for a single large multimodal vision-language-action (VLA) foundation model, as opposed to a general-purpose application. The core is an 8B-parameter model built on LLaMA-3-8B-Instruct with a ConvNeXt vision encoder, capable of understanding images/video and producing both textual responses and discretized robot/UI actions. The repository contains the full training stack: model definitions, data preprocessing, dataset-specific constructors, mixed RLDS/PyTorch data loaders, action tokenization, DeepSpeed training configs, and evaluation hooks for robotics benchmarks like LIBERO and SimplerEnv. It also ships FastAPI servers and demo agents for UI automation and robot trajectory planning, but these are secondary to the model training and data pipeline core, making this a hybrid repo with a dominant data-research character. The data lineage is central to the project. Raw heterogeneous datasets—Open X-Embodiment, Ego4D, EpicKitchen, SeeClick, LLaVA, and the in-house Magma-820K—are normalized into a unified multimodal conversation format via per-dataset Constructor subclasses. Visual point tracks from CoTracker2 are clustered, quantized, and rendered as 'Trace-of-Mark' overlays; bounding boxes and labels are rendered as 'Set-of-Mark' overlays. Continuous robot actions are discretized into token IDs aligned with the LLM vocabulary, allowing a standard causal LM objective to train action prediction alongside visual understanding. The resulting model can be deployed via the included FastAPI server for interactive inference. Reproducing the full pipeline requires access to gated or proprietary datasets, CoTracker2 checkpoints, a custom transformers fork, and significant GPU resources, so end-to-end reproduction is not turnkey, though inference on the released HuggingFace checkpoint is more accessible.
This repository contains the research code for UniDriveVLA, a unified Vision-Language-Action (VLA) model for autonomous driving. It couples a Qwen3-VL based understanding expert with sparse 3D perception and trajectory planning experts via a Mixture-of-Transformers design. The codebase supports two parallel experiment tracks: nuScenes (real-world open-loop evaluation) and Bench2Drive (CARLA closed-loop simulation). It includes full training pipelines (staged pretraining on driving QA datasets, then fine-tuning on nuScenes/Bench2Drive), evaluation harnesses for detection, motion, planning, and VQA, and a CARLA leaderboard integration for scenario-based testing. The repository is primarily a research artifact—it is not a production application but rather a comprehensive release of model definitions, training scripts, data converters, and evaluation metrics for reproducing the paper's results. Significant code duplication exists between the nuScenes and Bench2Drive directories, and many paths are hardcoded, reducing portability. The code demonstrates strong engineering for research (custom CUDA ops, distributed training with DeepSpeed, modular mmdet3d plugins) but lacks the polish of a reusable library.