thehardwarestore.ai

NVIDIA CUTLASS is a comprehensive C++ template library and Python DSL for implementing high-performance CUDA kernels, primarily focused on matrix multiply (GEMM), convolution, and attention operations. It provides a layered architecture from low-level warp matrix multiply-accumulate (MMA) and copy instructions (CuTe atoms) up through threadblock-level collective mainloops and epilogues, kernel-level schedulers (persistent, stream-K, warp-specialized), and a high-level operator API for runtime dispatch with fusion support. The repository includes a Python-embedded DSL (CuTeDSL) that uses MLIR-based compilation to generate optimized CUDA kernels, a separate Python code generation backend (cutlass_cppgen) for JIT compilation, and an extensive profiling and testing infrastructure. It spans NVIDIA GPU architectures from Volta to Blackwell and is actively maintained with contributions to deep learning frameworks like PyTorch and JAX.

NCCL is NVIDIA's Collective Communications Library, a fundamental building block for multi-GPU and multi-node computing. It provides highly optimized implementations of standard collective operations (all-reduce, all-gather, reduce, broadcast, reduce-scatter) and point-to-point communication primitives. The library features a plugin architecture for extensibility, supporting various network transports (InfiniBand, sockets, shared memory), GPU Direct technologies, and remote memory access. With language bindings for Python and Rust, and extensive diagnostic and tuning capabilities, NCCL serves as the communication backbone for deep learning frameworks and HPC applications requiring efficient inter-GPU coordination.

The NVIDIA Container Toolkit is a production-grade suite of command-line tools, OCI hooks, and libraries that enable GPU acceleration inside containers. It provides a container runtime shim (`nvidia-container-runtime`), a prestart hook (`nvidia-container-runtime-hook`), a CDI (Container Device Interface) hook binary (`nvidia-cdi-hook`), a comprehensive CLI (`nvidia-ctk`) for generating CDI specs and configuring container engines (Docker, containerd, CRI-O), and an installer daemon. The system discovers NVIDIA GPU devices, libraries, and capabilities using NVML, nvsandboxutils, and platform-specific logic (Tegra CSV, WSL DXCore), then injects them into container OCI specs via hooks or CDI. It supports multiple runtime modes (legacy, CSV, CDI, JIT-CDI) and handles special features like MIG, IMEX channels, CUDA forward compatibility, and graphics capabilities. The codebase is well-structured with clean interfaces, extensive test coverage (unit, integration, e2e), and is actively maintained by NVIDIA.

This repository is the NVIDIA cuQuantum SDK, a comprehensive suite of GPU-accelerated libraries for quantum computing simulation. It provides high-performance implementations for state-vector simulation (cuStateVec), tensor network contraction and decomposition (cuTensorNet), density matrix evolution (cuDensityMat), Pauli propagation (cuPauliProp), and stabilizer simulation (cuStabilizer). The codebase includes Python bindings with both high-level APIs (e.g., NetworkState, DenseMixedState, Operator) and low-level Cython bindings, as well as C++ sample codes, a benchmark suite, and JAX extensions for differentiable quantum operations. The architecture is modular, with each quantum simulation method organized into its own subpackage under `cuquantum/`. The Python package exposes a consistent interface for circuit simulation, operator construction, and distributed execution via MPI/NCCL. Key entry points include the `cuquantum.tensornet`, `cuquantum.densitymat`, `cuquantum.stabilizer`, and `cuquantum.pauliprop` modules, along with the `cuquantum.bindings` for direct C API access. The repository also contains extensive tests and samples that validate correctness against analytical results and reference implementations. The codebase is actively maintained with recent commits and issue triage. It demonstrates strong engineering practices with reusable patterns for memory management, callback-based operator coefficients, distributed tensor layouts, and custom JAX differentiation. However, some experimental APIs and version inconsistencies (e.g., CITATION.cff vs pyproject.toml) indicate areas of potential drift. Overall, it is a high-quality, production-ready SDK for quantum computing research and development.

This repository, NVlabs/tiny-cuda-nn, is a high-performance C++/CUDA library and PyTorch extension for training and querying neural networks with a focus on extreme GPU efficiency. It provides a modular, JSON-configurable framework of specialized components: input encodings (multiresolution hash grids, spherical harmonics, frequency, etc.), network architectures (fully fused MLP using Tensor Cores, CUTLASS-based MLP), loss functions, and optimizers (Adam, Shampoo, Novograd, etc.). The library supports runtime JIT compilation to fuse forward, backward, and loss kernels into single CUDA kernels, yielding up to 5x speedups. It is the engine behind NVIDIA's Instant Neural Graphics Primitives (Instant NGP) and is widely used in NeRF and neural rendering research. The codebase is production‑quality, with extensive use of templates, GPU memory arenas, CUDA graphs, multi‑stream parallelism, and comprehensive CI across many CUDA versions and GPU architectures. Although primarily a library, it includes sample applications and benchmarks that demonstrate its use in image overfitting and throughput measurement. PyTorch bindings via pybind11 make it accessible for Python‑based research workflows.

This repository implements a system to use NVIDIA GPU VRAM as swap space on Linux via the NBD (Network Block Device) protocol. The core component is a userspace daemon (nbd-vram) that allocates VRAM using the CUDA driver API and serves it as a block device over a Unix socket. The Linux kernel's NBD driver connects to this socket, exposing /dev/nbdX as a swap device. The system includes scripts for connection/disconnection, power management (automatically disabling swap on battery), and proper handling of suspend/resume to avoid GPU-related deadlocks. Key features include memory locking to prevent the daemon from being swapped out, using PR_SET_IO_FLUSHER to avoid deadlocks under memory pressure, and multi-threaded request batching to amortize CUDA synchronization overhead. The solution works with any CUDA-capable NVIDIA GPU without requiring kernel modifications or NVIDIA's P2P API, which is restricted to professional GPUs.

Sunone Aimbot is an AI-powered aimbot for first-person shooter games that uses YOLO models to detect players and heads in real-time screen captures. The system captures frames via MSS, BetterCam, or OBS, processes them with Ultralytics YOLO, identifies targets (prioritizing heads unless disabled), calculates mouse movement to aim at targets, and optionally shoots automatically. It features a modular architecture with separate threads for capture, visualization, overlay, shooting, and hotkey monitoring. A helper API and React-based UI provide configuration management, model training/exporting, and testing capabilities. The aimbot supports multiple input methods including standard mouse, GHUB, Razer, and Arduino devices.

Tilus is a tile-level GPU kernel programming language and compiler framework, primarily targeting NVIDIA architectures (SM80, SM90, SM100). It provides a Python DSL (`tilus.Script`) for writing high-performance CUDA kernels with explicit management of shared/register memory, tensor cores (WGMMA, tcgen05), and advanced synchronization primitives (mbarrier, cluster sync, TMA). The compiler stack includes a custom IR with layout inference (register/shared/tensor-memory), a pass pipeline (dead code elimination, bound-aware simplification, sub-byte type lowering), and backend emitters that generate PTX/CUDA code. Autotuning over tile sizes, warp counts, and pipeline stages is built into the framework. The architecture separates concerns into three layers: the user-facing `lang` package (Script, autotune, instructions), the `ir` package (tensors, layouts, instructions, inference rules), and the `backends` package (emitters, contexts, code generation). The `hidet` subpackage integrates a vendored copy of the Hidet IR and transforms, which provides the low-level expression/statement IR and a comprehensive set of optimization passes. Examples cover progressive matmul implementations (V0–V6) for Ampere, Hopper, and Blackwell, as well as flash attention, layer norm, and quantization kernels. This is a research project (ASPLOS '26 paper) with a strong emphasis on correctness and performance, evidenced by CUDA-level validation against PyTorch and cuBLAS. While the codebase is well-structured and demonstrates advanced compiler design patterns, it remains in beta (version 0.2.0) and carries some duplication between vendored Hidet code and native Tilus code, as well as hardware-specific constraints (NVGPU only). The layout inference system and autotuning infrastructure are particularly notable as reusable components.

NVIDIA Apex is a PyTorch extension library that provides high-performance, fused CUDA kernels for mixed-precision and distributed deep learning training. In production, it serves as a drop-in acceleration layer for PyTorch models, offering optimized implementations of common operations such as layer normalization, group normalization, fused optimizers (Adam, LAMB, SGD), and structured sparsity (2:4 pruning). The library is not an end-user application but a collection of reusable components that integrate seamlessly with PyTorch's autograd and distributed data-parallel frameworks. The architecture is a hybrid Python/C++/CUDA stack: Python modules expose user-facing APIs (e.g., `apex.optimizers.FusedAdam`, `apex.normalization.FusedLayerNorm`), while the heavy lifting is done by custom CUDA kernels compiled as PyTorch C++ extensions. Key entry points include the `apex.optimizers`, `apex.normalization`, and `apex.contrib` subpackages, which contain both stable and experimental features. Data flows are typical of PyTorch training loops: forward/backward passes invoke autograd Functions that dispatch to fused CUDA kernels, and distributed training utilities (e.g., `DistributedFusedAdam`) overlap gradient synchronization with computation. External dependencies include PyTorch, CUDA, cuDNN, and NCCL. The library also includes experimental modules for TorchInductor multi-stream scheduling, OpenFold-specific Triton kernels, and cuDNN frontend-based operations. While the core components are well-maintained, some contrib modules show signs of technical drift (duplicate optimizer implementations, deprecated API usage, hardcoded shape constraints). Overall, Apex is a mature, high-quality library that has been widely adopted in the PyTorch ecosystem for accelerating large-scale model training.

Alpamayo 1 is a Vision-Language-Action (VLA) model designed for autonomous driving applications. It implements the Alpamayo-R1 architecture that combines a Qwen3-VL backbone for vision-language reasoning with a diffusion-based action expert for trajectory prediction. The system takes multi-camera video and egomotion history as inputs, generates Chain-of-Causation (CoC) reasoning traces, and outputs future trajectories (6.4s horizon at 10Hz). Key components include a unicycle kinematic model for action space conversion, CUDA graph acceleration for efficient repeated inference, and robust geometric transformations. While the repository is noted to receive only limited maintenance as newer versions exist, the current implementation demonstrates strong engineering practices with several reusable components.

This repository contains a native WebRTC streaming application designed for Raspberry Pi and NVIDIA Jetson platforms. It captures video from various camera sources (libcamera, libargus, V4L2) and audio from PulseAudio/ALSA, encodes the streams using hardware acceleration when available or software fallbacks, and transmits them via WebRTC with multiple signaling options (MQTT, WebSocket, HTTP/WHEP). The application includes recording capabilities, snapshot functionality, and IPC for local control. The architecture is modular with clearly separated concerns for capture, encoding, signaling, and real-time communication.

GPU-T is a Linux desktop utility for comprehensive GPU information and monitoring, positioned as a GPU-Z alternative for Linux. It provides real-time sensor data (clocks, temperatures, power, fan speeds), static hardware specifications from JSON databases and sysfs, and detection of API capabilities (Vulkan, OpenCL, CUDA, etc.) through a modular probe system. The application follows the MVVM pattern with Avalonia UI and features a sidecar process for NVIDIA-specific NVAPI/NVML interactions to ensure stability and isolate native interop. Hardware detection relies on sysfs paths, external tool execution, and vendor-specific databases, with user override capabilities for hardware specifications. The modular design separates concerns into probes for hardware polling, advanced providers for feature detection, and utilities for common tasks like settings management and shell command execution.

NVIDIA Streamline is a middleware SDK that simplifies integrating NVIDIA graphics technologies—including DLSS (super resolution, frame generation, ray reconstruction), Reflex (low latency), NIS (image scaling), DeepDVC (dynamic contrast), and DirectSR—into DirectX 12, 11, and Vulkan games on Windows. It provides a unified API, a plugin-based architecture where each feature is a separate loaded module, and an interposer (hooking) layer that intercepts graphics API calls (DXGI, D3D12, Vulkan) to automatically inject features without requiring game engine modifications. The SDK includes a core infrastructure with parameter sharing, logging, thread management, OTA updates, security verification, and a compute abstraction layer. While primarily an application integration SDK, it is not a research project; all code is production-oriented middleware. The repository is well-structured with clear separation of concerns, but heavily depends on NVIDIA-specific SDKs (NGX, NVAPI, Reflex) and is Windows-only, limiting portability. Build system uses premake5 with a custom packman package manager.

Parakeet.cpp is a high-performance speech recognition inference library for NVIDIA's Parakeet model family (TDT-CTC, RNNT, Streaming EOU, Nemotron, Sortformer), written in C++ with C bindings. It provides offline and streaming transcription, speaker diarization, voice activity detection (VAD), beam search with optional n-gram language model fusion, phrase boosting, and timestamp extraction. The library is built on the axiom tensor library with Metal GPU acceleration for Apple Silicon, and supports audio input from WAV, FLAC, MP3, and OGG formats. The codebase includes a CLI tool, a Unix domain socket server example, and a rich set of examples covering all major features. While primarily an application/library for production ASR inference, it also contains research-oriented model conversion scripts (NeMo to safetensors) and a benchmark script comparing with PyTorch inference, giving it a hybrid character. The architecture is cleanly modular, with separate modules for audio I/O, VAD, model encoders/decoders, and decoding algorithms. The C API enables binding to Python, Swift, Rust, and other languages. Documentation and build system are well-maintained, and the repository is actively developed with recent commits.

This repository is a container build system and runtime for NVIDIA Jetson edge AI devices. It provides a modular, composable framework for building Docker images pre-configured with AI/ML libraries (LLMs, VLMs, robotics, speech, computer vision) optimized for Jetson's ARM64 + CUDA architecture. The system includes a CLI (`jetson-containers`), dependency resolution, multi-stage Docker builds, automated testing, and CI/CD workflow generation. While primarily a build tool, it also ships hundreds of ready-to-use container definitions for popular frameworks like PyTorch, TensorFlow, vLLM, ComfyUI, ROS 2, and Home Assistant. The repository has a hybrid character: the core is an application build system, but many package definitions are research-oriented (e.g., attention kernels, diffusion models, physical AI), reflecting its dual role as both a production deployment tool and an experimentation platform for edge AI.

The solo-cli is a command-line tool designed to deploy and manage Physical AI systems on local hardware. It integrates functionalities for serving large language models (via Ollama, vLLM, or llama.cpp), controlling robotic arms (through the LeRobot framework), authenticating with a custom model hub (Solo Hub), and fine-tuning models using external datasets. The application features a modular command structure with lazy-loaded dependencies to optimize startup time, cross-platform hardware detection for automated backend selection, and a layered configuration system. Additional utilities include Docker management, system information gathering, and filesystem-safe string handling for robotics data operations. The project also includes example implementations of Model Context Protocol (MCP) servers for various domains, demonstrating how to extend the CLI's capabilities for specialized AI workflows.

nvml-wrapper is a Rust workspace providing safe, high-level bindings to the NVIDIA Management Library (NVML). It is a library crate, not an application: there are no binaries, servers, or CLIs. The workspace is split into `nvml-wrapper-sys` (low-level FFI bindings, including runtime loading of `libnvidia-ml.so`/`nvml.dll` via `libloading`) and `nvml-wrapper` (a safe, typed API). The primary entry point is `Nvml::init()`, which loads the NVML library and returns a handle; from there, `Device` exposes 100+ methods for querying GPU state such as memory, clocks, temperature, power, ECC, NvLink, vGPU, MIG, and GPM metrics. The crate also supports mutating operations like setting application clocks and fan speed, and includes a `legacy-functions` feature for older NVML versions.

OptiScaler is a graphics enhancement tool that intercepts upscaler calls from games and redirects them to user-selected backends, enabling upscaler replacement (e.g., DLSS to FSR/XeSS) and frame generation. It functions as a DLL that hooks into DirectX 11/12 and Vulkan APIs to inject custom rendering behavior. The architecture is modular with clear separation between hooking systems, upscaler implementations, frame generation, low-latency technologies, and configuration management. Key entry points include dllmain.h (DLL entry point), hooking modules in hooks/, feature providers in upscalers/, and frame generation implementations in framegen/. The system relies heavily on external SDKs (NVIDIA NGX, AMD FidelityFX, Intel XeSS) and uses Detours for API interception. While well-structured with good abstraction patterns, the codebase shows signs of maintenance complexity due to supporting multiple APIs and upscaler versions, creating combinatorial complexity in testing and compatibility.

Firemote is a Home Assistant custom dashboard card that emulates physical remote controls for a wide range of media devices including Amazon Fire TV, Apple TV, Chromecast, NVIDIA Shield, Roku, Xiaomi, and more. The card provides realistic visual remotes with extensive customization: over 330 app launcher shortcuts, button overrides via Home Assistant scripts or actions, multiple remote styles, and support for device-specific compatibility modes. It integrates with Home Assistant's Android Debug Bridge, Android TV Remote, Apple TV, and Roku integrations. The repository contains the compiled JavaScript card and supporting data files for device definitions and translations, but the source is distributed as a single minified bundle. It is actively maintained with a large community and frequent updates.