This repository is the official code companion to the book '50 ML projects to understand LLMs' by Mike X Cohen. It contains over 50 Jupyter notebooks organized into chapters (2–7), each exploring a specific concept in large language model interpretability and internals. Topics span tokenization, embeddings, cosine similarity, softmax/logits, loss functions, sampling, hidden state analysis, effective dimensionality, logit lens, attention mechanisms, MLP projections, probing classifiers, and bias measurement. The code is purely educational and experimental—no production services, APIs, or deployable applications. All notebooks are designed to run on Google Colab with minimal setup, using standard libraries (PyTorch, Hugging Face Transformers, scikit-learn, numpy, matplotlib) and pretrained models (GPT-2, BERT, RoBERTa, Pythia). Each project has a 'helper' notebook (incomplete code for students) and a 'solution' notebook (complete reference implementation). The repository is a valuable resource for learning mechanistic interpretability and hands-on LLM analysis, but it lacks modularization, dependency pinning, and test coverage, making it unsuitable for direct reuse in production pipelines.
This repository is the official companion code for the O'Reilly book 'Natural Language Processing with Transformers' by Tunstall, von Werra, and Wolf. It contains 11 Jupyter notebooks, one per chapter, covering the full spectrum of NLP with Transformers: text classification, named entity recognition, text generation, summarization, question answering, model compression, few-shot learning, and training a transformer from scratch. The code is built on the Hugging Face ecosystem (transformers, datasets, accelerate) and is designed to be run interactively on Colab, Kaggle, or local GPU environments. The notebooks are well-structured with clear section headings, explanatory text, and visualizations, making them an excellent educational resource for learning state-of-the-art NLP techniques. Although the repository is primarily educational, it includes a SageMaker adaptation for some chapters, demonstrating deployment of Hugging Face models on AWS. The codebase is not intended as a production-grade library; rather, it serves as a tutorial and reference for practitioners. The overall quality is solid, with clean, reproducible code patterns, but the reliance on pinned, outdated library versions (e.g., transformers 4.16.2, datasets 1.16.1) and the deprecated Chapter 7 (farm-haystack) indicate moderate technical drift. The concepts remain relevant, but the exact code may require adjustments to run with the latest versions.
BLURR is a library that integrates Hugging Face transformers with fastai for NLP tasks. It provides high-level, mid-level, and low-level APIs to train and deploy transformer models for text classification, token classification, question answering, summarization, translation, and language modeling. The library leverages fastai's data block API and learner system, wrapping Hugging Face models to work seamlessly within the fastai training loop. Key components include custom data blocks for text processing, model wrappers for Hugging Face integration, specialized loss functions, and callbacks for optimization. The repository includes extensive examples and notebooks demonstrating usage with standard datasets like GLUE, SQuAD, and CNN/DailyMail.
bertviz is a Python library for interactive visualization of attention in Transformer-based NLP models. It provides three complementary views: Head View (per-head attention patterns), Model View (attention flow between tokens across layers), and Neuron View (query/key interactions within individual attention heads). The library is designed for Jupyter notebooks, rendering D3.js-based interactive HTML directly in cell output. It accepts attention tensors from most HuggingFace models via output_attentions=True, and includes a vendored, modified copy of older HuggingFace transformers code to expose query/key vectors for the Neuron View on BERT, GPT-2, RoBERTa, XLNet, and similar architectures.
This repository is a massive collection of Jupyter notebooks, course materials, and documentation for the Hugging Face ecosystem (Transformers, Diffusers, Datasets, PEFT, smolagents, etc.). It serves as an educational resource, containing the official Hugging Face course in multiple languages, auto-generated documentation notebooks, and community examples covering NLP, computer vision, audio, and generative AI. It also includes SageMaker integration examples for training and deployment on AWS. The repository is primarily a documentation and learning resource, not a production application. It exhibits hybrid nature as it includes both educational content and reusable utility scripts and training patterns. The code quality is typical for tutorials: functional but not production-grade, with some deprecated API usage (e.g., `load_metric`), translation drift across languages, and auto-generated notebooks that may lag behind library updates. However, the core educational content is well-structured and actively maintained by Hugging Face.
This repository is a curated collection of Jupyter notebooks and Python scripts that demonstrate fine-tuning, inference, quantization, and optimization techniques for state-of-the-art vision and multimodal models from the Hugging Face ecosystem. It covers a wide range of tasks including visual question answering, object detection, image classification, video understanding, multimodal retrieval, and preference alignment (DPO). The code is primarily educational, designed to be run interactively in Google Colab, with each notebook or script serving as a self-contained tutorial for a specific model or technique. Notable patterns include LoRA/QLoRA fine-tuning, knowledge distillation via a custom ImageDistilTrainer, any-to-any embedding for retrieval, and video frame downsampling for multimodal input. The repository is not a production library or application; it is a learning resource for engineers and researchers exploring vision-language models.
This repository is not a software application, data pipeline, or research codebase. It is a content-distribution repository for a single educational artifact: a 'VIP cheatsheet' PDF condensing Stanford CME 295 (Transformers & Large Language Models). The repo contains 16 language folders, each holding one PDF and one translated README, plus a root README and an MIT license. There is zero executable code, no tests, no CI/CD, no dependency manifests, and no data processing logic of any kind. It is classified as data_research only by process of elimination — binary data files dominate and there are no application entry points — but in reality it is educational content, a hybrid that fits neither software nor data-engineering categories cleanly.
This repository is an educational collection of minimal, from-scratch implementations of core LLM and generative AI systems, maintained by DataWhale. It is not a single production application; rather, it is a set of self-contained 'Tiny' sub-projects covering a small LLM trained on TinyStories, a RAG pipeline, a ReAct agent, an LLM evaluation harness, a DDPM diffusion model, a Transformer, a GraphRAG system, and an image-based RAG pipeline. Each sub-project has its own README, requirements, and runnable code or notebooks, with heavy Chinese-language comments explaining the underlying theory. The code is intentionally simple and readable rather than production-hardened, and it serves as a teaching resource for understanding how these systems work internally.
IPEX-LLM (Intel Extension for PyTorch for Large Language Models) is a library and toolkit designed to accelerate LLM inference and fine-tuning on Intel hardware (CPU, GPU, NPU) through low-bit quantization (INT4, FP8, etc.), hardware-specific optimizations, and seamless integration with popular LLM ecosystems. The core functionality centers around the `optimize_model` function and optimized model loaders that enable efficient INT4/FP8 inference with minimal code changes. It provides integrations with Hugging Face Transformers, vLLM, LangChain, LlamaIndex, and serving solutions like FastAPI and FastChat. Despite its utility for LLM acceleration on Intel platforms, the project is explicitly archived with no active maintenance, updates, or security patches, limiting its long-term viability.
This repository is a comprehensive educational resource focused on large language models (LLMs), reinforcement learning (RL), and related training algorithms. It contains over 100 original diagrams (SVG/PNG/PDF) covering topics such as LLM architecture, SFT, DPO, RLHF, PPO, GRPO, policy gradient, and reasoning optimization. The repo also includes a curated index of open-source LLMs and VLMs with links to papers, code, and configs. While not a traditional software application or data pipeline, it serves as a visual knowledge base for researchers and practitioners in AI alignment and model training. The only executable code is a few utility scripts for image processing; the main value lies in the meticulously crafted diagrams that explain complex concepts with clarity. The repo is actively maintained and frequently updated with new content.