This repository is the official code companion to the book '50 ML projects to understand LLMs' by Mike X Cohen. It contains over 50 Jupyter notebooks organized into chapters (2–7), each exploring a specific concept in large language model interpretability and internals. Topics span tokenization, embeddings, cosine similarity, softmax/logits, loss functions, sampling, hidden state analysis, effective dimensionality, logit lens, attention mechanisms, MLP projections, probing classifiers, and bias measurement. The code is purely educational and experimental—no production services, APIs, or deployable applications. All notebooks are designed to run on Google Colab with minimal setup, using standard libraries (PyTorch, Hugging Face Transformers, scikit-learn, numpy, matplotlib) and pretrained models (GPT-2, BERT, RoBERTa, Pythia). Each project has a 'helper' notebook (incomplete code for students) and a 'solution' notebook (complete reference implementation). The repository is a valuable resource for learning mechanistic interpretability and hands-on LLM analysis, but it lacks modularization, dependency pinning, and test coverage, making it unsuitable for direct reuse in production pipelines.
bertviz is a Python library for interactive visualization of attention in Transformer-based NLP models. It provides three complementary views: Head View (per-head attention patterns), Model View (attention flow between tokens across layers), and Neuron View (query/key interactions within individual attention heads). The library is designed for Jupyter notebooks, rendering D3.js-based interactive HTML directly in cell output. It accepts attention tensors from most HuggingFace models via output_attentions=True, and includes a vendored, modified copy of older HuggingFace transformers code to expose query/key vectors for the Neuron View on BERT, GPT-2, RoBERTa, XLNet, and similar architectures.