$ show modular/llm-inference-handbook
This repository is the LLM Inference Handbook, a comprehensive technical documentation site built with Docusaurus. It serves as an educational resource for engineers building, optimizing, and deploying LLM inference systems. The site covers inference fundamentals, model preparation, interaction patterns, optimization techniques (batching, KV cache, prefix caching, speculative decoding), kernel optimization (FlashAttention, GPU architecture), and infrastructure operations. It includes interactive React components such as GPU memory calculators, KV cache simulators, batching visualizers, and GPU execution visualizers that demonstrate concepts through live demos. The codebase is well-structured with TypeScript, CSS modules, a custom design system (DLS), and integration with Mantine UI. While primarily a documentation site, the interactive components are self-contained and could be extracted for reuse in other educational contexts. The handbook is framework-agnostic but has a strong emphasis on the Modular MAX ecosystem.
The LLM identified 4 files worth highlighting for reuse:
A self-contained React component that renders an interactive GPU die diagram showing streaming multiprocessors (SMs), warps, threads, L2 cache, and HBM. It animates execution flow and memory access patterns, making GPU architecture concepts tangible. The component uses CSS modules for styling and is decoupled from the rest of the handbook, making it reusable in any educational context about GPU computing.
An interactive component that visualizes GPU memory hierarchy (registers, shared memory, L1/L2 cache, HBM) with adjustable parameters. It demonstrates latency and bandwidth trade-offs. The component is self-contained with its own CSS module and state management, suitable for extraction into other GPU education materials.
A reusable form input component with a label, tooltip description, and consistent styling. Used across all calculator components (GPU Memory, KV Cache, Quantization, Latency Metrics). It encapsulates a common pattern for labeled numeric inputs with validation and tooltip support, making it a good candidate for extraction into a shared UI library.
An interactive GPU memory calculator that estimates memory requirements for LLM inference based on model size, quantization, batch size, and sequence length. It provides real-time feedback and visual breakdowns. The component is self-contained with its own CSS module and uses the FormItem pattern for inputs. It could be adapted for other memory estimation tasks.