$ show rasbt/reasoning-from-scratch
This repository is the official companion code for the book "Build a Reasoning Model (From Scratch)" by Sebastian Raschka (Manning, 2025). It is primarily a data_research codebase focused on building, evaluating, and training reasoning LLMs from scratch using the Qwen3 0.6B model as a base. The code covers inference-time scaling (temperature/top-p sampling, self-consistency), reinforcement learning from verifiable rewards (GRPO/RLVR), self-refinement, and knowledge distillation from larger teachers (DeepSeek-R1, Qwen3 235B). It includes a full custom Qwen3 implementation with KV cache, RoPE, and grouped-query attention, plus training scripts, notebooks, benchmark evaluations (MATH-500, MMLU), and Hugging Face export adapters. While educational in nature, it demonstrates production-quality engineering for research experiments. The repository is hybrid: it provides a reusable Python package for text generation and training, but its dominant character is data_research due to its notebook-driven, experimental, and benchmark-focused workflow.
The LLM identified 6 files worth highlighting for reuse:
Hybrid LaTeX parser that extracts \boxed{} answers with high accuracy. Implements both a fast heuristic fallback and an ANTLR-based grammar for edge cases (e.g., nested boxes, conditionals, multicolumn). Self-contained with no internal package imports; could be reused in any math QA pipeline.
Contains `top_p_filter` – a clean, self-contained PyTorch implementation of nucleus (top-p) filtering with proper renormalization, edge-case handling (guaranteed at least one token), and no internal package dependencies. Reusable in any generation pipeline.
`self_consistency_vote` function that samples multiple answers, extracts final candidates, performs majority voting with tie-breaking (via heuristic or logprob scores), and returns detailed statistics. Modular design accepts arbitrary generation and scoring functions; reusable for any generative benchmark requiring plurality voting.
`download_file` function with primary and backup URLs, retries, and progress reporting. Also contains `check_model_file` and `download_from_github` – robust data loading patterns that handle network failures and missing files gracefully. Standalone and reusable for any project needing reliable model or dataset downloading.
`compute_grpo_loss` implements the GRPO objective with PPO-clipping, advantage normalization, and group-based importance sampling. Modular with clear separation of loss components (policy ratio, clip, KL). Comments document known numerical nuances (e.g., correct torch.minimum vs torch.where). Reusable for any GRPO/RLVR training pipeline.
`build_examples` and `load_distill_data` from the distillation pipeline: converts raw teacher outputs (JSON with prompt/answer fields) into tokenized training examples with optional thinking tokens, length filtering, and train/val split. Cleanly separates data loading, formatting, and filtering stages; reusable for any distillation project.