$ show changyeyu/LLM-RL-Visualized
This repository is a comprehensive educational resource focused on large language models (LLMs), reinforcement learning (RL), and related training algorithms. It contains over 100 original diagrams (SVG/PNG/PDF) covering topics such as LLM architecture, SFT, DPO, RLHF, PPO, GRPO, policy gradient, and reasoning optimization. The repo also includes a curated index of open-source LLMs and VLMs with links to papers, code, and configs. While not a traditional software application or data pipeline, it serves as a visual knowledge base for researchers and practitioners in AI alignment and model training. The only executable code is a few utility scripts for image processing; the main value lies in the meticulously crafted diagrams that explain complex concepts with clarity. The repo is actively maintained and frequently updated with new content.
The LLM identified 2 files worth highlighting for reuse:
Contains a self-contained PyTorch implementation of the LoRA (Low-Rank Adaptation) algorithm as a nn.Module class. The code demonstrates the core low-rank decomposition, initialization strategy (kaiming uniform for A, zeros for B), scaling factor, and dropout. It is a clean, isolated pattern that can be extracted and reused in any PyTorch project requiring parameter-efficient fine-tuning.
Contains a standalone Python function for Generalized Advantage Estimation (GAE). The function computes advantages from rewards and value estimates using a simple recursive loop over time steps. It is parameterized by gamma and lambda, making it directly reusable in any RL training pipeline without modification.