This repository is the official companion code for the book "Build a Reasoning Model (From Scratch)" by Sebastian Raschka (Manning, 2025). It is primarily a data_research codebase focused on building, evaluating, and training reasoning LLMs from scratch using the Qwen3 0.6B model as a base. The code covers inference-time scaling (temperature/top-p sampling, self-consistency), reinforcement learning from verifiable rewards (GRPO/RLVR), self-refinement, and knowledge distillation from larger teachers (DeepSeek-R1, Qwen3 235B). It includes a full custom Qwen3 implementation with KV cache, RoPE, and grouped-query attention, plus training scripts, notebooks, benchmark evaluations (MATH-500, MMLU), and Hugging Face export adapters. While educational in nature, it demonstrates production-quality engineering for research experiments. The repository is hybrid: it provides a reusable Python package for text generation and training, but its dominant character is data_research due to its notebook-driven, experimental, and benchmark-focused workflow.
Kiln is a full-stack AI development platform for building, evaluating, and optimizing LLM-based tasks and agents. It provides a desktop application (Tkinter tray + FastAPI server) with a Svelte web UI, plus a core Python library and a cloud server component. The system supports the complete AI workflow: project/task management, multi-provider LLM orchestration via LiteLLM, evaluation with multiple judge types (LLM-as-judge, code eval, exact match, pattern match, etc.), synthetic data generation, RAG pipelines (chunking, embedding, vector stores), fine-tuning, prompt optimization, and Git-based project versioning. It also includes a 'Kiln Pro' cloud service for advanced features like copilot-assisted spec refinement and batch generation. The architecture is a monorepo with uv workspace, featuring extensive test coverage (unit, integration, e2e) and CI/CD pipelines. While the primary character is an application, it incorporates significant data engineering patterns for evaluation pipelines, synthetic data generation, and statistical analysis.
Vibe Check MCP is a production-ready MCP (Model Context Protocol) server that provides metacognitive oversight for AI agents. It implements the research-backed Chain-Pattern Interrupt (CPI) methodology to prevent reasoning lock-in by injecting reflective pauses during agent workflows. The server exposes six tools: `vibe_check` for generating clarifying questions via a secondary LLM, `vibe_learn` for logging patterns of mistakes and successes, and three constitution management tools (`update_constitution`, `reset_constitution`, `check_constitution`) for per-session behavioral rules. It supports multiple LLM providers (Gemini, OpenAI, Anthropic, OpenRouter) with automatic fallback to static questions when the LLM is unavailable, and maintains session history for context-aware feedback. The architecture is a monolithic MCP server with dual transport modes: stdio for local agent clients (Claude Desktop, Cursor, Windsurf, VS Code) and streamable HTTP for remote access. A CLI tool (`vibe-check-mcp`) automates installation into all major MCP client configurations with atomic writes and backup creation, handles secret management via interactive prompts or env files, and provides diagnostic commands. The server includes robust HTTP security hardening: DNS-rebinding protection via Host header validation, loopback-only CORS by default with a configurable allowlist, explicit JSON body size limits, and proper JSON-RPC error responses instead of HTML pages. The codebase is well-organized with separation of concerns, extensive unit and integration tests (over 30 test files), CI/CD with security scanning, and clean abstraction layers such as a central model registry and request-scoped transport wrappers.
Eko is a TypeScript framework for building production-ready AI agents that can automate workflows across browser, Node.js, and web extension environments. The core orchestration engine (Eko class) takes a natural language task, uses an LLM to generate a multi-agent workflow plan encoded as XML, then executes it with parallel/serial agent execution, replanning, and human-in-the-loop support. The system provides a clean abstraction over LLM providers (OpenAI, Anthropic, etc.) with retry and fallback, a ReAct tool-calling loop, and built-in browser automation agents that operate via screenshots or DOM labels. It also includes MCP client implementations (SSE, HTTP, stdio) for integrating external tools, and a chat agent for conversational interactions with tool delegation. The architecture is modular with separate npm packages: @eko-ai/eko-core (core engine), @eko-ai/eko-nodejs (Playwright-based browser agent), @eko-ai/eko-extension (Chrome extension agent), and @eko-ai/eko-web (web environment agent). The repository includes working examples (Chrome extension with side panel, Node.js automation using Playwright, and a React web app). The framework is designed for developers who want to automate complex, multi-step tasks using language models, with a focus on reliability, human oversight, and extensibility via custom agents and tools.