$ account OpenHelix-Team
[github ↗]
1 repos indexed
The repository implements LLaVA-VLA, a Vision-Language-Action model for robotic manipulation tasks. It extends the LLaVA framework to predict action sequences from visual inputs, proprioceptive state, and language instructions using a diffusion-based action head. The system includes training pipelines for fine-tuning on datasets like CALVIN, inference servers (Flask and Gradio) for real-time robot control, and evaluation tools for benchmarks. The architecture features modular vision encoders, projectors, and resamplers with factory patterns for flexibility, action tokenization for discrete action prediction, and integration with PEFT/DeepSpeed for efficient training.
RATING:
MAINT: active
DRIFT: low