$ show mikexcohen/ML4LLM_book
This repository is the official code companion to the book '50 ML projects to understand LLMs' by Mike X Cohen. It contains over 50 Jupyter notebooks organized into chapters (2–7), each exploring a specific concept in large language model interpretability and internals. Topics span tokenization, embeddings, cosine similarity, softmax/logits, loss functions, sampling, hidden state analysis, effective dimensionality, logit lens, attention mechanisms, MLP projections, probing classifiers, and bias measurement. The code is purely educational and experimental—no production services, APIs, or deployable applications. All notebooks are designed to run on Google Colab with minimal setup, using standard libraries (PyTorch, Hugging Face Transformers, scikit-learn, numpy, matplotlib) and pretrained models (GPT-2, BERT, RoBERTa, Pythia). Each project has a 'helper' notebook (incomplete code for students) and a 'solution' notebook (complete reference implementation). The repository is a valuable resource for learning mechanistic interpretability and hands-on LLM analysis, but it lacks modularization, dependency pinning, and test coverage, making it unsuitable for direct reuse in production pipelines.
The LLM identified 1 files worth highlighting for reuse:
This notebook demonstrates a clean, reusable pattern for extracting MLP activations from GPT-2 via forward hooks and training an XGBoost classifier to probe syntactic categories. The `implant_hook` function (defined inline) registers a hook on a specific layer's MLP activation and stores outputs in a dictionary. This pattern can be extracted and adapted to any transformer model for activation-based probing or feature analysis. The notebook also includes a laminar sweep loop and an ablation hook for causal intervention, making it a self-contained template for mechanistic interpretability experiments.