tinygrad is a deep learning framework that provides a full-stack solution from tensor operations and automatic differentiation to a custom compiler stack that generates optimized GPU kernels for multiple backends (CUDA, HIP, Metal, WebGPU, etc.). It includes a JIT compiler (TinyJit), a universal operation IR (UOp) for graph rewriting and custom kernel authoring, and a suite of neural network layers, optimizers, and datasets. The repository also contains extensive MLPerf benchmark implementations, a LLM server with streaming, and low-level hardware interaction tools (AMD firmware flashing, GPU profiling). It is primarily a software library for training and inference, with a strong research component in kernel optimization and hardware exploration.
This repository implements generative recommendation models, primarily HSTU (Hierarchical Sequential Transduction Units) and DLRM-v3, for large-scale sequential recommendation. It is a hybrid codebase combining research experimentation (ICML'24 paper on generative recommenders) with production-grade inference benchmarking (MLPerf loadgen integration). The code provides custom high-performance CUDA and Triton kernels for flash attention, jagged tensor operations, and layer normalization, targeting modern NVIDIA GPUs (Hopper/Blackwell). It includes training pipelines with gin configuration for datasets like MovieLens, Amazon Books, KuaiRand, and synthetic streaming data up to 100B items. The repository also contains a full MLPerf inference benchmark suite with vendored loadgen library and end-to-end TorchScript tracing for deployment. While the core focus is research on scaling laws and architecture innovations for sequential recommenders, the ops and modules are engineered to production standards with multiple kernel backends and rigorous tests.