MiniMind is an educational open-source toolkit for training small language models entirely from scratch using native PyTorch. It provides a complete pipeline including custom BPE tokenizer training, pretraining, supervised fine-tuning (SFT), LoRA, RLHF (DPO), RLAIF (PPO/GRPO/CISPO), agentic reinforcement learning, and knowledge distillation. The system is designed to be accessible (low compute cost, ~3 RMB and 2 hours on a single GPU) and includes inference interfaces such as a CLI, Streamlit web UI, and an OpenAI-compatible API server. While primarily a software application for training and deploying LLMs, it also contains dataset preparation, evaluation benchmarks, and research experiments, giving it a hybrid character that leans toward application due to its focus on providing reusable training scripts and deployment tools.
OpenRLHF is a production-oriented distributed RLHF/RL training framework for large language models, not a research notebook collection. The codebase implements a full training stack: SFT, reward modeling, DPO/IPO/cDPO, and RL algorithms including PPO, REINFORCE++, GRPO, RLOO, Dr.GRPO, and FlashREINFORCE. Its architecture combines Ray for distributed actor orchestration and placement groups, vLLM for high-throughput rollout generation, DeepSpeed ZeRO for model training, and HuggingFace Transformers for model handling. The primary entry points are CLIs under openrlhf/cli, with train_ppo_ray.py as the main RL launcher; these accept a large set of hierarchically parsed dotted flags that control training, rollout, reward, vLLM, and DeepSpeed behavior. The repository is genuinely hybrid in spirit because it trains research-scale models and reproduces academic methods, but its dominant character is application/framework: it ships packaging, Dockerfiles, CLI entry points, unit tests, and versioned releases. The design emphasizes a token-in-token-out agent abstraction that unifies single-turn and multi-turn RLHF, plus sync and async training modes, a hybrid engine that colocates training and generation on shared GPUs, and VLM support. Distributed checkpointing, LoRA/QLoRA, packing, dynamic batch sizing, importance sampling corrections, and KL-estimator variants are all present in the code, not merely in the README. Craftsmanship is strong. The code separates concerns across cli/, trainer/, models/, datasets/, and utils/, and includes several genuinely reusable patterns such as a self-contained math answer grader, a flat-to-hierarchical argparse config converter, a typed RL experience dataclass, and distributed loss-normalization utilities. The test suite exercises loss estimators, trainer control flow, experience batching, and agent behavior with mocked heavy dependencies, which is a pragmatic way to test GPU-bound code on CPU. The main weaknesses are operational rather than architectural: dependencies are extremely tightly pinned (deepspeed==0.19.6, transformers==5.15.0, ray==2.55.0, vllm==0.29.0), and the code monkey-patches upstream internals and relies on version-specific behaviors. This makes upgrades brittle, and the CI pipeline builds wheels without running the test suite. Overall, this is a mature, actively maintained framework with high reusability for engineers building LLM alignment pipelines, though it demands GPU infrastructure and careful dependency alignment.
nanochat is a minimal, hackable LLM training and inference framework focused on reproducing GPT-2 performance quickly on 8×H100 GPUs. It provides an end-to-end pipeline from tokenizer training and dataset preparation through pretraining, supervised fine-tuning, reinforcement learning (GRPO/REINFORCE), and a chat CLI. The core is a highly optimized GPT model with advanced features including a custom MuonAdamW hybrid optimizer, FP8 training with tensorwise dynamic scaling, unified Flash Attention 3/SDPA interface, sliding window attention, value embeddings, and BOS-aligned best-fit packing dataloader. Hyperparameters are automatically derived from scaling laws based on model depth, enabling compute-optimal training. The codebase is research-oriented, with extensive experiment logging, leaderboard tracking, and scaling law analysis notebooks. While primarily an application (CLI-driven training/evaluation framework), it has strong data_research elements including dataset pipelines, evaluation tasks (CORE, MMLU, GSM8K, etc.), and analytical notebooks.
Gitleaks is a command-line secret detection tool written in Go that scans for sensitive information like API keys, tokens, and passwords in git repositories, directories, files, and stdin. It features a modular architecture with separate components for configuration loading, source abstraction (file/git/archive handling), detection logic with multi-layer decoding, rule-based matching, and flexible reporting. The tool uses a Cobra-based CLI interface with commands for different scan modes and supports output formats including JSON, CSV, JUnit, SARIF, and custom templates. Key strengths include its pluggable decoder pipeline for handling encoded secrets, configurable rule system with entropy checks and keyword filtering, and sophisticated allowlist mechanism for reducing false positives. While actively maintained for security patches, the project is feature-complete with ongoing dependency management challenges due to its extensive rule set and reliance on unstable packages like golang.org/x/exp.
H2O LLM Studio is a web-based application for fine-tuning large language models (LLMs) without requiring coding experience. It provides a GUI built on H2O Wave and CLI tools for dataset import, experiment configuration, model training, evaluation, and export to Hugging Face Hub. The system supports multiple problem types including causal language modeling, classification, regression, sequence-to-sequence, and Direct Preference Optimization (DPO). The architecture centers around a Wave server handling both UI and backend logic, with distributed training capabilities via DeepSpeed. Key components include a configuration system using dataclasses, utilities for conversation chain handling, model loading abstractions, and factory patterns for training components.
This repository provides a minimal, from-scratch implementation of a Vision-Language Model (VLM) called MiniMind-V, designed for educational purposes and low-resource training. It ingests the ALLaVA-4V dataset (Parquet format with image bytes and JSON conversations) for pretraining (1.27M caption pairs) and supervised fine-tuning (2.9M mixed instruct/caption/text). The architecture follows a LLaVA-1.5 style: a frozen SigLIP2 visual encoder (256×256, 64 patch tokens) projects via a 2-layer MLP into a MiniMind LLM (with optional Mixture-of-Experts). Training freezes the visual encoder and trains only the projector and selected LLM layers, achieving ~3 RMB / 2 hours on a single RTX 3090. The codebase includes training scripts (pretrain and SFT), evaluation, a Gradio web demo, and model conversion to HuggingFace Transformers format. While primarily a research/educational project, it demonstrates a clean, modular VLM pipeline that can be adapted for other multimodal tasks.
PMetal is a comprehensive machine learning framework specifically optimized for Apple Silicon hardware. It provides a complete suite for LLM training, fine-tuning, inference, and deployment through multiple interfaces (CLI, TUI, GUI, Python SDK, and REST API). The framework leverages Apple-specific technologies like Metal GPU, ANE, and UltraFusion for performance optimization while supporting standard ML workflows including LoRA/QLoRA fine-tuning, knowledge distillation, model merging, and quantization to formats like GGUF. The architecture follows a modular workspace design with over 20 specialized crates handling different aspects of the ML lifecycle, unified through a job specification system and configuration framework.
This repository is an extensive, open-source educational tutorial on building and training large language models (LLMs) from scratch, maintained by the Datawhale community. It covers the full pipeline: NLP fundamentals, Transformer architecture (with a custom Tiny-K implementation), pretraining, supervised fine-tuning (SFT), parameter-efficient fine-tuning (LoRA), reinforcement learning (GRPO, OPD, Search-R1, ReTool), and practical applications like RAG and Agent systems. The code is organized into chapters with clear progression, and each chapter includes runnable scripts, configuration files, and detailed documentation in Chinese. While primarily educational, the repository contains several well-crafted, reusable components such as a modular Transformer implementation, efficient dataset loaders, a RAG pipeline, an Agent framework, and RL training loops with distributed support. The project is actively maintained with recent additions (Extra-Chapter) and has a large community following. However, reproducibility is moderate due to fragmented dependency files, hardcoded paths, and reliance on external services (e.g., DeepSeek Search, SwanLab). The code is not production-grade but serves as an excellent learning resource and a source of reusable patterns for LLM research and development.
LLM Foundry is a production-grade framework for training, finetuning, evaluating, and serving large language models, with a focus on the MPT architecture and HuggingFace integration. It provides a full pipeline from data preparation (MDS streaming), to model definition (custom MPT, HF wrappers, embedding models), to distributed training (Composer, FSDP, TP), and to evaluation (ICL benchmarks, gauntlets). The architecture is modular, using YAML-driven configuration and a central registry pattern for extensibility. The codebase includes dozens of callbacks for monitoring and training stabilization, custom optimizers, and efficient data collation (bin packing). While tightly integrated with MosaicML's Composer and platform, the core registry and layer abstractions are reusable. The repository is actively maintained with comprehensive test suites, though some features remain experimental and several dependencies (flash attention, megablocks) require careful version management.
SDPO (Self-Distilled Policy Optimization) is a framework for training large language models using reinforcement learning algorithms, built upon the verl (Volcano Engine Reinforcement Learning) foundation. The system provides a modular architecture for implementing PPO, GRPO, SDPO, and other RL algorithms with distributed training capabilities via Ray. It integrates with inference engines like vLLM and SGLang, supports multiple model architectures (Llama, Qwen with Megatron optimization), and includes reward modeling for domains like mathematics and coding. The framework features extensible components for reward managers, tools, and agent interactions, with experimental capabilities for fully asynchronous training and multi-turn reasoning. While actively maintained, the system's complexity and extensive dependency ecosystem present moderate technical drift risks.
Megatron-MoE-ModelZoo is a production-ready toolkit for training Mixture-of-Experts (MoE) large language models including DeepSeek-V3, Qwen3, and Mixtral. It provides SLURM job submission scripts, Dockerfiles for hardware-optimized containers, YAML configuration files, and best practice guides for launching training jobs on HPC clusters. The repository integrates with NVIDIA's Megatron-LM framework (dev branch) to enable efficient distributed training of MoE models across tensor, pipeline, and expert parallelism dimensions. Key features include precision selection (BF16/FP8/MXFP8), checkpoint conversion utilities, and memory estimation tools for hardware planning.
llm-action is a comprehensive resource hub for large language model engineering that combines theoretical explanations with practical implementation code. It covers the full LLM lifecycle including data engineering, training (full fine-tuning and parameter-efficient methods like LoRA/QLoRA), model compression (quantization, pruning, distillation), inference serving, deployment, and evaluation. The repository contains both educational documentation and runnable code examples, with particular emphasis on Chinese-language resources and domestic AI hardware adaptation (especially Huawei Ascend NPUs). While it serves as a knowledge base, its inclusion of actual training scripts, inference servers, and utility tools makes it primarily application-oriented rather than purely research-focused.
LLaVA-NeXT is a large multimodal model (LMM) research codebase that extends the LLaVA family to support single-image, multi-image, video, and interleaved tasks. It provides training pipelines for supervised fine-tuning (SFT), direct preference optimization (DPO), and reinforcement learning (PPO, GRPO) with custom vision encoders (CLIP, SigLIP, EVA-CLIP, MLCD) and modular projector/resampler components. The repository also includes a full RL training framework (EasyR1/verl) for large-scale GRPO training using Ray, FSDP, and vLLM, as well as evaluation scripts for video and interleaved benchmarks. While primarily a research codebase with extensive training configurations and experimental scripts, it also contains serving infrastructure (Gradio, SGLang, controller-worker) for deployment. The code is well-structured with reusable vision encoder wrappers, data protocols, and RL algorithm implementations, but suffers from hardcoded internal paths, multiple overlapping RL implementations, and legacy training pipelines that increase maintenance burden.
This repository is an educational book/course (DataWhale 'diy-llm') that teaches building and training LLMs from scratch. It spans the full pipeline: tokenizer, Transformer architecture, GPU high-performance programming, scaling laws, data engineering, distributed training, alignment (including GRPO/RLVR), evaluation, and multimodal models. The content is primarily Chinese, with Jupyter notebooks and code snippets embedded in markdown chapters. It is not a production library or application; it is a tutorial/textbook with homework assignments.
Open-R1 is a research-focused application for reproducing DeepSeek-R1's training and evaluation pipeline. It provides scripts for supervised fine-tuning (SFT) and group relative policy optimization (GRPO), along with tools for synthetic data generation using teacher models, dataset filtering based on pass rates, and evaluation on reasoning and code benchmarks (AIME, MATH, GPQA, LiveCodeBench). The project is a collection of CLI scripts and Python modules rather than a deployed service. It integrates tightly with Hugging Face Hub, Weights & Biases, and external code execution providers (E2B, Morph, Piston) for reward computation. The codebase exhibits good modularity in its reward functions and code execution provider abstractions, but is now deprecated in favor of upstream TRL. The repo is primarily a training/evaluation application, though aspects like dataset filtering and decontamination make it hybrid with data-research characteristics.