GEM is a Gym-like framework for training and evaluating agentic LLMs via reinforcement learning. It provides a diverse set of environments (games, math, code, QA, etc.) that can be used to train LLMs to interact with tools, solve problems, and make sequential decisions. The framework features a modular design with composable wrappers for observation modification, tool integration, and episode tracking, along with vectorized environments for parallel execution. It supports integration with multiple RL training frameworks (Oat, Verl, Tinker, OpenRLHF, RL2) through example scripts, making it a versatile tool for LLM agent research.
SDPO (Self-Distilled Policy Optimization) is a framework for training large language models using reinforcement learning algorithms, built upon the verl (Volcano Engine Reinforcement Learning) foundation. The system provides a modular architecture for implementing PPO, GRPO, SDPO, and other RL algorithms with distributed training capabilities via Ray. It integrates with inference engines like vLLM and SGLang, supports multiple model architectures (Llama, Qwen with Megatron optimization), and includes reward modeling for domains like mathematics and coding. The framework features extensible components for reward managers, tools, and agent interactions, with experimental capabilities for fully asynchronous training and multi-turn reasoning. While actively maintained, the system's complexity and extensive dependency ecosystem present moderate technical drift risks.
This repository is an educational book/course (DataWhale 'diy-llm') that teaches building and training LLMs from scratch. It spans the full pipeline: tokenizer, Transformer architecture, GPU high-performance programming, scaling laws, data engineering, distributed training, alignment (including GRPO/RLVR), evaluation, and multimodal models. The content is primarily Chinese, with Jupyter notebooks and code snippets embedded in markdown chapters. It is not a production library or application; it is a tutorial/textbook with homework assignments.