$ show huggingface/open-r1
Open-R1 is a research-focused application for reproducing DeepSeek-R1's training and evaluation pipeline. It provides scripts for supervised fine-tuning (SFT) and group relative policy optimization (GRPO), along with tools for synthetic data generation using teacher models, dataset filtering based on pass rates, and evaluation on reasoning and code benchmarks (AIME, MATH, GPQA, LiveCodeBench). The project is a collection of CLI scripts and Python modules rather than a deployed service. It integrates tightly with Hugging Face Hub, Weights & Biases, and external code execution providers (E2B, Morph, Piston) for reward computation. The codebase exhibits good modularity in its reward functions and code execution provider abstractions, but is now deprecated in favor of upstream TRL. The repo is primarily a training/evaluation application, though aspects like dataset filtering and decontamination make it hybrid with data-research characteristics.
The LLM identified 4 files worth highlighting for reuse:
Contains a collection of reward functions for GRPO training, including accuracy, format, cosine-scaled, repetition penalty, length-based, and code execution rewards. Each function is self-contained and follows a consistent signature, making them easily reusable across different training loops or projects. The module also includes a registry factory (`get_reward_funcs`) that dynamically builds the reward list from script arguments. High craftsmanship in separation of concerns and extensibility.
Defines an abstract base class `CodeExecutionProvider` and concrete implementations for E2B and MorphCloud sandboxes. The factory function `get_provider` selects the appropriate provider based on configuration. This design allows easy swapping of code execution backends without modifying training code. The providers handle parallel execution, timeouts, and error recovery. A clean, reusable pattern for integrating cloud code interpreters.
A package for evaluating code submissions against competitive programming problem sets (IOI and CodeForces). It includes scoring logic for subtasks with caching, test batching, and multiple scoring modes (pass/fail, partial, weighted sum). Uses async Piston and Morph clients for sandboxed execution, with automatic retry and load balancing. The modular design allows reuse across different code-evaluation tasks.
Defines hierarchical dataclasses for training script arguments, including support for dataset mixtures with weights, column selection, and test splits. Custom `__post_init__` validation ensures consistency (e.g., column alignment across mixed datasets). Provides a clean, type-safe way to configure experiments via YAML files. Reusable as a configuration pattern for other ML training projects.