This repository is the official companion code for the book "Build a Reasoning Model (From Scratch)" by Sebastian Raschka (Manning, 2025). It is primarily a data_research codebase focused on building, evaluating, and training reasoning LLMs from scratch using the Qwen3 0.6B model as a base. The code covers inference-time scaling (temperature/top-p sampling, self-consistency), reinforcement learning from verifiable rewards (GRPO/RLVR), self-refinement, and knowledge distillation from larger teachers (DeepSeek-R1, Qwen3 235B). It includes a full custom Qwen3 implementation with KV cache, RoPE, and grouped-query attention, plus training scripts, notebooks, benchmark evaluations (MATH-500, MMLU), and Hugging Face export adapters. While educational in nature, it demonstrates production-quality engineering for research experiments. The repository is hybrid: it provides a reusable Python package for text generation and training, but its dominant character is data_research due to its notebook-driven, experimental, and benchmark-focused workflow.
This repository is the official companion code for the O'Reilly book 'Natural Language Processing with Transformers' by Tunstall, von Werra, and Wolf. It contains 11 Jupyter notebooks, one per chapter, covering the full spectrum of NLP with Transformers: text classification, named entity recognition, text generation, summarization, question answering, model compression, few-shot learning, and training a transformer from scratch. The code is built on the Hugging Face ecosystem (transformers, datasets, accelerate) and is designed to be run interactively on Colab, Kaggle, or local GPU environments. The notebooks are well-structured with clear section headings, explanatory text, and visualizations, making them an excellent educational resource for learning state-of-the-art NLP techniques. Although the repository is primarily educational, it includes a SageMaker adaptation for some chapters, demonstrating deployment of Hugging Face models on AWS. The codebase is not intended as a production-grade library; rather, it serves as a tutorial and reference for practitioners. The overall quality is solid, with clean, reproducible code patterns, but the reliance on pinned, outdated library versions (e.g., transformers 4.16.2, datasets 1.16.1) and the deprecated Chapter 7 (farm-haystack) indicate moderate technical drift. The concepts remain relevant, but the exact code may require adjustments to run with the latest versions.