$ show datawhalechina/tiny-universe
This repository is an educational collection of minimal, from-scratch implementations of core LLM and generative AI systems, maintained by DataWhale. It is not a single production application; rather, it is a set of self-contained 'Tiny' sub-projects covering a small LLM trained on TinyStories, a RAG pipeline, a ReAct agent, an LLM evaluation harness, a DDPM diffusion model, a Transformer, a GraphRAG system, and an image-based RAG pipeline. Each sub-project has its own README, requirements, and runnable code or notebooks, with heavy Chinese-language comments explaining the underlying theory. The code is intentionally simple and readable rather than production-hardened, and it serves as a teaching resource for understanding how these systems work internally.
The LLM identified 5 files worth highlighting for reuse:
Implements a complete GraphRAG pipeline: entity and relationship extraction via LLM, graph construction, Leiden community detection, and local/global query modes. The design is backend-agnostic through abstract LLM and embedding interfaces, making it a reusable template for building knowledge-graph RAG systems on different document collections.
Provides a compact DDPM noise scheduler and diffusion process implementation, covering forward noising and reverse denoising. This is a self-contained, well-structured component that can be extracted and reused for any diffusion-based generative modeling experiment.
Contains reusable NLP evaluation metrics including F1, Rouge, and classification accuracy, used to score LLM predictions across multiple datasets. The functions are generic and can be dropped into any LLM evaluation harness with minimal adaptation.
Implements a ReAct-style agent loop that interleaves reasoning and tool calls with an LLM. The pattern is self-contained and demonstrates a clean separation between the agent loop, LLM backend, and tool registry, making it a useful reference for building tool-using agents.
A from-scratch encoder-decoder Transformer with configurable multi-head attention, including optional Flash Attention support and sinusoidal positional encoding. It is a clear reference implementation that can be extracted for educational purposes or as a starting point for custom Transformer architectures.