clpeak is a cross-platform synthetic micro-benchmark that measures peak achievable compute performance (FLOPS/TOPS), bandwidth, latency, and AI inference throughput across CPUs, GPUs, and NPUs. It supports eight hardware backends: OpenCL, Vulkan, CUDA, ROCm/HIP, Metal, oneAPI/SYCL, native CPU (with per-ISA dispatch), and ONNX Runtime. The tool is architected as a set of backend-specific static libraries orchestrated by a common framework, with two front-ends: a CLI tool (`clpeak`) and a Flutter desktop/mobile GUI. The GUI communicates with the native library via a C-ABI FFI bridge, streaming structured events for live display. Results are serialized to a versioned JSON schema (v3) and can be compared against baselines. The codebase is notable for its meticulous anti-optimization engineering (MAD chains, compiler barriers, calibration probes) and its protobuf-free ONNX model generator. It is a mature, actively maintained project with extensive CI coverage and packaging for multiple platforms.
Lemonade is a local AI inference server that provides an OpenAI-compatible API for LLM chat, completions, embeddings, image generation, TTS, audio transcription, and 3D model generation. It features a multi-backend architecture supporting over 20 inference engines (llama.cpp, vLLM, Stable Diffusion, Kokoro, Moonshine, ONNX Runtime, etc.) with automatic model management, intelligent routing policies, and a desktop GUI built with Tauri and React. The system includes a CLI tool for model management and benchmarking, a web app for browser-based interaction, and extensive platform support (Linux, Windows, macOS, Docker). Key architectural patterns include a plugin-based backend system with declarative descriptors, a routing policy engine with classifier-based and deterministic rules, a streaming proxy for SSE forwarding, and platform abstraction layers for processes, paths, and system metrics. The codebase demonstrates high craftsmanship with well-isolated components, comprehensive testing, and attention to cross-platform compatibility. While primarily an application, it also includes data-oriented components such as model metadata extraction from GGUF files, benchmarking infrastructure, and telemetry collection for inference monitoring.
AITemplate is a high-performance deep neural network inference compiler developed by Meta. It takes model graphs defined via a PyTorch-like `nn.Module` API (or auto-converted from PyTorch FX via `fx2ait`) and compiles them into optimized CUDA or HIP GPU kernels using CUTLASS and Composable Kernel libraries. The compiler performs extensive operator fusion (GEMM + activation, convolution + elementwise, attention), kernel profiling to select optimal tile configurations, and memory planning to produce a standalone shared library with no runtime dependencies on vendor math libraries. The system currently targets NVIDIA SM80+ and AMD CDNA2 architectures, supporting models like ResNet, BERT, Vision Transformer, Stable Diffusion, and Detectron2. The codebase is predominantly an application—a compiler framework—rather than a data research project, though it incorporates research-level kernel fusion and attention patterns.
DeePMD-kit is a comprehensive, production-grade deep learning package for many-body interatomic potential energy representation and molecular dynamics simulation. It provides a unified framework for training and deploying machine learning interatomic potentials (MLIPs) across multiple backends: TensorFlow, PyTorch, JAX, and PaddlePaddle, with a backend-agnostic core (dpmodel) that ensures numerical consistency. The package supports a wide range of descriptor architectures (SeA, SeR, SeT, DPA1/2/3/4, attention-based, hybrid) and fitting targets (energy, force, virial, dipole, polarizability, density of states, spin, property). It integrates tightly with LAMMPS via custom pair styles and plugins, and also supports ASE, i-PI, GROMACS, and other MD engines. The codebase is highly modular, with a plugin registration system for models, descriptors, and losses, and offers extensive serialization capabilities for model export and compression. Pretrained models (DPA-3, DPA-4) are available for direct download and fine-tuning. The project is actively maintained with a large community, comprehensive test suites, and continuous integration across CPU and GPU platforms.
Kineto is a production-grade C++ library for GPU and CPU performance profiling, primarily embedded in PyTorch's profiler. It provides low-overhead timeline tracing of GPU kernels, memory operations, synchronization events, and CPU-side activities, with support for CUDA (CUPTI), ROCm (rocprofiler-sdk), and Intel XPU (PTI) backends. The library supports both synchronous (manual start/stop) and asynchronous (on-demand, daemon-triggered) profiling modes, and emits Chrome Trace JSON and in-memory trace buffers for downstream analysis. It is not a standalone application; it is a linkable library with a public C API and plugin interfaces for custom profilers and loggers. Architecturally, Kineto is organized around a central ActivityProfilerController that drives a warmup/collect/process state machine, delegating platform-specific capture to backend profilers derived from GenericActivityProfiler. Configuration flows through a ConfigLoader that polls files or an IPC fabric daemon, and activity data flows from vendor callback/activity APIs through correlation-ID-based flow tracking into typed metadata and trace spans. The codebase demonstrates strong software engineering practices: extensive unit tests, CI matrices for CPU/CUDA/ROCm/XPU, clean interface abstractions, and reusable patterns such as observer-based logging, factory-based output registration, and type-safe metadata visitors. However, heavy conditional compilation, vendor SDK version guards, and platform-specific ifdefs introduce maintenance complexity and moderate technical drift risk.