This repository is the official code companion to the book '50 ML projects to understand LLMs' by Mike X Cohen. It contains over 50 Jupyter notebooks organized into chapters (2–7), each exploring a specific concept in large language model interpretability and internals. Topics span tokenization, embeddings, cosine similarity, softmax/logits, loss functions, sampling, hidden state analysis, effective dimensionality, logit lens, attention mechanisms, MLP projections, probing classifiers, and bias measurement. The code is purely educational and experimental—no production services, APIs, or deployable applications. All notebooks are designed to run on Google Colab with minimal setup, using standard libraries (PyTorch, Hugging Face Transformers, scikit-learn, numpy, matplotlib) and pretrained models (GPT-2, BERT, RoBERTa, Pythia). Each project has a 'helper' notebook (incomplete code for students) and a 'solution' notebook (complete reference implementation). The repository is a valuable resource for learning mechanistic interpretability and hands-on LLM analysis, but it lacks modularization, dependency pinning, and test coverage, making it unsuitable for direct reuse in production pipelines.
This repository implements and evaluates Function Vectors (FVs) for steering large language models (LLMs) without fine-tuning. The core workflow involves computing task-conditioned mean head activations from in-context learning examples, identifying causally influential attention heads via indirect effect, summing their outputs to form the FV, and applying it during inference to elicit target behaviors (e.g., translation, inflection). The system provides a modular Python library with utilities for model loading, prompt construction, activation extraction, intervention, and evaluation, accompanied by a demo notebook and evaluation scripts for running experiments on various linguistic datasets. While the codebase demonstrates clean separation of concerns and contains reusable components for model intervention and prompt handling, it shows signs of technical drift due to dependency rigidity and architecture-specific hooks, and appears to have stagnated in maintenance with no recent updates.