$ show rohitg00/ai-engineering-from-scratch
This repository is a structured, comprehensive AI engineering curriculum spanning 20 phases and over 500 lessons, from math foundations and ML fundamentals to deep learning, computer vision, NLP, speech, transformers, generative AI, reinforcement learning, LLMs from scratch, agent engineering, multi-agent systems, production infrastructure, and ethics/safety. It is an educational resource, not a production application or data research project. The code consists of self-contained pedagogical implementations (primarily Python, with some TypeScript, Rust, and Julia) that demonstrate core algorithms from scratch—such as attention mechanisms, BPE tokenization, diffusion models, RLHF, and agent loops—using synthetic data and deterministic stubs. Capstone projects include small web servers (Hono/TypeScript) and test harnesses. The curriculum is actively maintained with CI tooling, certification prep materials, and translation support. While the code is simplified for learning, it exhibits clean patterns, consistent lesson structure, and testing practices that make individual components (e.g., tool registries, state machines, evaluation harnesses) reusable in other contexts.
The LLM identified 5 files worth highlighting for reuse:
Implements a reusable ToolRegistry with JSON Schema validation for tool definitions, structured error handling, and parallel dispatch. The pattern is self-contained, uses only stdlib, and can be extracted to build function-calling infrastructure for any LLM agent project. Demonstrates non-obvious craftsmanship in handling tool schemas, dispatch errors, and result normalization.
A clean, from-scratch implementation of Byte-Pair Encoding (BPE) tokenization, including merge learning from a corpus, encoding, and decoding. Uses Python stdlib, is well-commented, and can be reused as a teaching tool or base for custom tokenizer experiments. The algorithm is non-trivial and implemented with clear data structures (dictionaries of merges, symbol frequencies).
Provides a MultiServerClient that manages multiple MCP sessions in-process, with JSON-RPC 2.0 dispatch for tools, resources, and prompts. Includes session lifecycle, merging tool lists from multiple servers, and error propagation. This is a prototypical MCP client scaffold that could be extracted to build real MCP integrations with subprocess or HTTP transport.
Implements a release-gate evaluation harness that compares model outputs against expected criteria, computes scores, and enforces thresholds. Uses dataclass-based configuration, pluggable metrics, and generates a structured report. This pattern is reusable for any evaluation pipeline needing test-driven quality gates for LLM outputs.
Demonstrates a complete RAG pipeline with hybrid retrieval (BM25 + dense embeddings) fused via reciprocal rank fusion (RRF), followed by a cross-encoder reranker. Includes chunking, embedding generation, and response generation with a mock LLM. The pipeline is modular and could be adapted for production RAG systems with real embedding models and rerankers.