simdjson is a high-performance C++ JSON parsing library that uses SIMD instructions (AVX2, AVX-512, NEON, RVV, etc.) to parse and validate JSON at multi-gigabyte-per-second speeds. It provides two primary APIs: a DOM API (`simdjson::dom`) that builds a complete in-memory tree, and an On-Demand API (`simdjson::ondemand`) that performs lazy, forward-only parsing for streaming and large-document use cases. The library also includes a builder API for JSON serialization, compile-time JSON parsing via C++26 reflection, JSON Pointer/Path support, NDJSON/JSON-sequence streaming, and a single-header amalgamated distribution. The codebase is organized around a two-stage pipeline: stage 1 performs SIMD-accelerated structural indexing and UTF-8 validation, and stage 2 builds the DOM tape or drives the on-demand iterator. Runtime CPU dispatch selects the best implementation for the host architecture, with fallbacks for non-SIMD platforms. The repository is a mature, production-grade library used by major projects (Node.js, ClickHouse, etc.) and is backed by an extensive test suite, fuzzing harnesses, and cross-architecture CI. It is primarily an application/library rather than a data-research repository, though it contains benchmark and research-oriented components (e.g., performance modeling, static reflection experiments).
Box2D is a mature, production-grade 2D rigid-body physics engine written in C17, designed primarily for game development. It provides a comprehensive C API for simulating rigid bodies, shapes (circles, capsules, polygons, chain segments), joints (revolute, prismatic, distance, weld, wheel, motor, etc.), sensors, and continuous collision detection (CCD). The engine is data-oriented, with a focus on determinism, multithreading, and SIMD acceleration (SSE2, AVX2, NEON). It includes a built-in scheduler for parallel constraint solving using graph coloring, and supports large worlds via double-precision coordinates. The architecture is modular: the public API is defined in `include/box2d/` with separate headers for types, math, collision, and the main `box2d.h`. The implementation in `src/` is organized into subsystems: world management, body/shape/joint creation, broad-phase collision (dynamic AABB tree), narrow-phase collision (GJK, SAT, manifold generation), constraint solving (contact solver with SIMD variants), island management, and recording/replay for deterministic debugging. The engine also provides snapshot serialization for save/load. While the core is a library, the repository includes a rich set of sample applications (using GLFW, OpenGL, and ImGui) that demonstrate every feature, a comprehensive unit test suite, and performance benchmarks. The codebase is exceptionally well-crafted, with clean abstractions, extensive validation, and a strong emphasis on correctness and performance. It is actively maintained by the original author (Erin Catto) and the community.
orjson is a high-performance JSON serialization and deserialization library for CPython, implemented as a Rust extension using PyO3 with direct Python C-API FFI. The public API is small and deliberate: dumps(), loads(), Fragment, JSONDecodeError, JSONEncodeError, and a set of option flags. Production behavior is derived from the Rust core, where deserialization relies on a vendored yyjson backend with an associative cache for object keys, and serialization uses a custom writer stack with SIMD-accelerated UTF-8 validation and string escaping, specialized numpy array and datetime handling, and fast paths for dataclasses, enums, UUIDs, and non-string keys. Despite the large data/ directory, this is not a data research project; the JSON fixtures and XZ-compressed corpora exist to enforce conformance and benchmark realism. The architecture is a classic C-extension pattern: src/lib.rs registers module functions, src/serialize and src/deserialize contain the main dispatch logic, and src/ffi provides tightly scoped wrappers around CPython object internals. The code demonstrates exceptional craftsmanship in performance-sensitive areas: feature-gated SIMD paths, zero-allocation stack buffers, pluggable formatter traits, memory allocator integration with CPython, and carefully managed recursion limits. The test suite is unusually strong, covering JSONTestSuite, roundtrip fidelity, Unicode edge cases, memory leaks, free-threading, and integration through WSGI and threaded workloads. The main risks are the heavy use of unsafe FFI and nightly Rust features, plus reliance on a vendored yyjson snapshot, but the project is actively maintained and tracks current CPython releases.
ncnn is a high-performance neural network inference framework designed for mobile, embedded, and desktop deployment. It provides a self-contained C++ library with no mandatory external dependencies, supporting CPU inference with extensive SIMD optimizations (ARM NEON, x86 SSE/AVX/AVX512, RISC-V Vector, MIPS MSA, LoongArch LSX/LASX) and GPU inference via Vulkan. The framework includes a comprehensive set of neural network layers (convolution, pooling, normalization, activation, recurrent, attention, etc.) and supports multiple data types (float32, float16, bfloat16, int8). The companion pnnx tool converts models from PyTorch (TorchScript) and ONNX into ncnn format using a sophisticated graph rewriting pipeline. Python bindings and a model zoo are provided for rapid prototyping. The codebase is production-grade, used in Tencent applications like QQ and WeChat, and is actively maintained with support for modern architectures and operations (e.g., scaled dot-product attention, rotary embeddings).
glam-rs is a high-performance 3D mathematics library for Rust, focused on game development and graphics applications. It provides vector, matrix, quaternion, and affine transformation types with optional SIMD acceleration across multiple architectures (SSE2, NEON, WASM, portable SIMD). The library emphasizes zero-cost abstractions, compile-time optimizations, and a consistent API through code generation and trait-based design. Key features include swizzle operations for flexible vector component access, camera matrices for LH/RH coordinate systems, and extensive optional dependencies for serialization, random number generation, and interoperability. The architecture separates concerns into numeric-type modules (f32, f64, integer vectors), SIMD backends, and feature-gated functionality. Code generation via Tera templates ensures consistency across types and operations while reducing boilerplate. The swizzle system (src/swizzles/vec_traits.rs) implements a reusable trait-based pattern for vector component manipulation that enables generic code over different vector implementations. Camera modules use const generics to parameterize handedness, depth range, and Y-axis orientation, allowing compile-time selection of graphics API conventions without runtime overhead. Maintenance is active with regular updates, comprehensive CI/CD pipelines, and a clear MSRV policy (1.68.2). The library employs automated benchmarking (gungraun) and coverage tracking to monitor performance and quality. While the feature flag system and multiple SIMD backends increase complexity, they are managed through strict versioning, dependency auditing, and code generation workflows, resulting in low technical drift. The project demonstrates strong engineering practices with extensive documentation, attribution to source inspirations, and a focus on cross-platform determinism.
The KFR library is a high-performance C++ framework for digital signal processing (DSP) and audio applications. It provides optimized implementations of FFT/DFT, filter design (FIR/IIR), sample rate conversion, audio file I/O (supporting WAV, FLAC, MP3, AIFF, CAF, etc.), SIMD-accelerated mathematical operations, and tensor/multi-dimensional array operations. The library is designed for real-time processing and scientific computing, with architecture-specific optimizations for x86 SIMD, ARM NEON, and RISC-V RVV, and runtime dispatch for multiarchitecture support. Key architectural features include a header-only design (with optional separate compilation), expression templates for lazy evaluation and operator fusion, a portable SIMD abstraction layer, and a tensor system with NumPy interoperability. The library is used in production by scientific collaborations such as LIGO, Virgo, and KAGRA for gravitational-wave signal processing, benchmarked against FFTW. It employs a dual licensing model (GPLv2+/commercial) and requires C++20 for compilation. The codebase demonstrates strong engineering practices through reusable components such as the SIMD vector abstraction (vec<T, N>), expression template system, tensor class, and audio data handling unit. These components are self-contained, exhibit non-obvious skill in C++ template metaprogramming, and can be extracted for use in other projects with minimal modification. The library is actively maintained, with recent commits and CI/CD pipelines, and shows low technical drift by keeping pace with modern C++ standards and hardware advancements.
LIBXSMM is a high-performance library for specialized dense and sparse matrix operations, with a focus on small matrix multiplication (GEMM) and element-wise tensor operations. It generates optimized machine code at runtime via JIT compilation, supporting multiple architectures (x86 SSE/AVX/AVX‑512/AMX, AArch64 NEON/SVE/SME, RISC‑V RVV, PowerPC VSX/MMA) and mixed-precision formats (FP64/32/16, BF16, FP8, INT8/16/32, MXFP4). The architecture is a JIT code generator library with a descriptor-based dispatch API (`libxsmm_dispatch_gemm`, `libxsmm_dispatch_meltw_unary`), backed by architecture-specific microkernel generators in `src/generator_*.c`. Key entry points include the public header `include/libxsmm.h`, the CPU feature detection (`libxsmm_cpuid_*.c`), and the binary export tool for offline kernel generation. Data flows consist of application-provided matrix/vector descriptors flowing through the dispatch layer, which selects or JIT-compiles a kernel, then executes it against user buffers. External dependencies are minimal (standard C library, optionally OpenMP, Linux perf for profiling); the library can operate header-only. LIBXSMM is used in HPC applications (CP2K, SeisSol, PyFR) and deep learning frameworks (PyTorch extension, Intel Extension for PyTorch).
This repository implements Stwo, a high-performance STARK proof system based on Circle STARKs (ePrint 2024/278) and the Stwo whitepaper. It is a Rust library providing a prover and verifier for zero-knowledge proofs over the Mersenne prime field M31 (p = 2^31 − 1) with extension fields CM31 and QM31. The architecture is modular: a core crate defines field arithmetic, FRI, polynomial commitment, and verification logic; a prover crate provides backend-agnostic proving with CPU and SIMD (AVX2/AVX-512, NEON, WASM) implementations; a constraint-framework crate offers a DSL for building AIR constraints and lookup arguments (LogUp, GKR). The system supports multiple hash channels (Blake2s, Poseidon252, Keccak256) and includes a no_std verifier suitable for on-chain deployment. Key entry points are `prover::prove()` and `core::verifier::verify()`. External dependencies include blake2, blake3, sha3, keccak, and starknet-crypto. The codebase is used in production by StarkWare's SHARP shared prover and by the Cairo VM prover (stwo-cairo). It demonstrates exceptional craftsmanship in SIMD optimization, field arithmetic, and protocol implementation, though it relies on nightly Rust and contains some unsafe code for performance.
This repository implements a C-based SQLite loadable extension that adds vector search capabilities directly within SQLite databases. It enables similarity search using distance metrics (L2, Cosine, Dot, L1, Hamming) on various vector types (F32, F16, BF16, I8, U8, BIT) without requiring virtual tables. The core architecture is a modular C extension with runtime SIMD dispatch for hardware-accelerated distance calculations, supporting platforms including Linux, macOS, Windows, Android, iOS, and RISC-V. The project provides official language bindings for Python, Node.js, Swift, Flutter, and Android, facilitating integration into diverse application stacks. It includes a TurboQuant implementation for compact 2/3/4-bit quantization, allowing for reduced memory footprint and approximate search with high recall. The codebase demonstrates high craftsmanship in low-level optimization, particularly in its SIMD intrinsics handling and cross-platform binary distribution strategy. While the repository contains benchmarking scripts and quantization research documentation, its primary function is as a production-ready library for edge AI and mobile workloads. It is designed for offline, on-device AI with a low memory footprint (~30MB default). The zero open issues and recent commit activity indicate a stable and actively maintained codebase with low technical drift.
The Dext Framework is a comprehensive, modular full-stack framework for Delphi (v11+) designed to bring modern web development patterns to the native Delphi ecosystem. It is Apache 2.0 licensed and provides a coherent set of libraries covering web API construction (minimal APIs, MVC controllers, middleware pipeline, SignalR-compatible real-time hubs), an ORM with code-first migrations, dependency injection, background jobs, event bus, resilience pipeline, testing/mocking utilities, desktop UI patterns (MVU/MVVM with VCL binding), AI integration (MCP server, agent graph), CLI tooling, and telemetry/dashboard. The architecture is heavily inspired by ASP.NET Core/EF Core, with fluent builders (`TDextServices`, `TApplicationBuilder`), specification pattern, attribute-based configuration, and a focus on zero-allocation performance (custom SIMD-optimized collections, ring-buffer logging, direct codecs). It includes extensive documentation (The Dext Book in English and Portuguese), benchmark harnesses, and 40+ example applications covering real-world use cases from simple CRUD to multi-tenant e-commerce. The codebase is actively maintained, with multi-version Delphi support (XE2 through 13), community database drivers (UniDAC), and a growing ecosystem of IDE integration and design-time tooling.
KISS FFT is a mixed-radix Fast Fourier Transform library implemented in C and C++. It provides a compact, portable, and reasonably efficient FFT implementation supporting complex and real-valued transforms in one or multiple dimensions. The library is designed to be simple to integrate, offering both a C API (kiss_fft, kiss_fftr, kiss_fftnd, kiss_fftndr) and a header-only C++ template class. It supports multiple scalar types (float, double, int16_t, int32_t) and SIMD acceleration via SSE, LSX, and LASX instruction sets. The core algorithm uses decimation-in-time with optimized butterflies for radices 2, 3, 4, and 5, and provides a cache mechanism (kfc) to reuse FFT configurations. The codebase is self-contained with no external dependencies beyond libm; optional tools require fftw3 and libpng for benchmarking and image generation. It is widely used in embedded systems, audio processing, and scientific computing due to its permissive BSD license and straightforward API.
fff (Fast File Finder) is a high-performance file search and content grep toolkit written in Rust, designed for long-running applications like code editors, AI agents, and file explorers. It provides fuzzy filename search with typo resistance, live content grep (plain text, regex, fuzzy), directory search, glob filtering, and git-aware frecency ranking. The core library maintains a persistent in-memory index with a bigram-based content prefilter, an LMDB-backed frecency and query history store, and a live filesystem watcher for incremental updates. The system is exposed through multiple interfaces: a Neovim plugin with a full TUI picker UI, an MCP server for AI assistants (Claude Code, Codex), language bindings for Python, Node.js, and Bun, and a C FFI for embedding in other languages. The architecture is a Rust core library with multiple frontends communicating via FFI or direct Rust API, all sharing the same `FilePicker` instance that manages indexing, searching, and watching. The codebase demonstrates exceptional performance engineering with SIMD-optimized string operations, compressed inverted indexes, arena-based path storage, and careful concurrency design with thread pools and lock-free snapshots.
Gorse is a production-grade, open-source recommender system engine written in Go. It implements a master/server/worker distributed architecture: the master coordinates training and cluster state, workers generate offline recommendations, and servers expose a REST API, dashboard, and gRPC endpoints. The system supports collaborative filtering via BPR and ALS matrix factorization, click-through rate prediction via an Attentional Factorization Machine, item-to-item/user-to-user similarity based on embeddings, non-personalized leaderboards, external script recommenders, and LLM-based reranking/chat recommenders. It includes a complete data model for users, items, feedback, and labels, with storage backends for SQL databases, MongoDB, Redis, and vector stores (xvec, Qdrant, Weaviate, Milvus), plus blob storage for model artifacts. The codebase is organized around strong abstractions: storage interfaces, protobuf-based services, a recommendation logic layer, model training/evaluation packages, and a common utility layer containing an autograd tensor library, SIMD-optimized float kernels, HNSW vector search, and concurrency helpers. It exposes admin/client CLIs, JSONL import/export, hyperparameter optimization with Optuna, monitoring metrics, and OpenTelemetry tracing. Although this repository contains ML training, benchmarks, and data-loading code, it is fundamentally an application platform rather than a data-research repository. The engineering is strong but carries complexity from supporting many storage backends, architecture-specific SIMD generated code, and a large monolithic master task file; some deprecated APIs and generated-code drift are present.
columnar is a Rust library for columnar representation of Rust data types. It provides a `Columnar` trait and a derive macro that transform ordinary structs and enums into columnar storage: each field is stored in its own typed vector, and enums store tags and payloads separately. The crate supports recursive/tree-structured enums, all-unit enums, `no_std` environments, and optional serde serialization. It is a library, not an application, with no web or CLI surface; the primary entry points are the derive macro and the `Columnar`/`ColumnarVec` traits.
Corrfunc is a high-performance scientific computing library for cosmology, specifically for computing two-point correlation functions and the void probability function from galaxy catalogs. It provides both a C library/command-line interface and Python bindings through C extensions. The code implements pair counting for real-space (DD, xi, wp), redshift-space (DDrppi, DDsmu), angular (DDtheta), and mock/survey geometries, with support for periodic simulation boxes and line-of-sight survey data. The architecture is a hybrid C core with Python wrappers: OpenMP parallelized C kernels, runtime dispatch across SSE4.2/AVX/AVX2/AVX512 intrinsics, grid-based cell decomposition, and a custom fast-food binary format for fast I/O. The project also includes a substantial test suite with reference outputs, CI workflows, and benchmark scripts used in a peer-reviewed performance paper. Despite being packaged as an application/library, the repository contains research artifacts: benchmark .npz files, paper scripts, and astrophysical mock-catalog test data. These are support materials for validating and advertising performance, not the primary deliverable. The dominant character is clearly a reusable scientific library, not a data-analysis pipeline. The codebase is mature and well-engineered for its niche, but shows some drift: Python 2 compatibility remnants, a complex dual Makefile/setup.py build, heavy code duplication between theory and mock modules, and platform-specific SIMD code that requires careful maintenance. It remains actively maintained and is the de facto standard CPU tool for fast correlation functions in cosmology.
The FastPFOR library is a C++ implementation focused on high-speed integer compression, particularly optimized for decoding 32-bit integers in data-intensive applications. It achieves performance through SIMD acceleration (SSE/AVX2/NEON) and asymmetric design prioritizing decompression speed over compression. The library provides a variety of codecs (FastPFor variants, SimdGroupSimple, VarInt, Simple8b/16, etc.) accessible via a factory pattern, enabling efficient storage and retrieval of integer arrays in scenarios like inverted indexes and columnar databases. Architecture centers around an abstract IntegerCODEC interface with concrete implementations for each codec, managed by CODECFactory. Low-level utilities include bitpacking helpers for arbitrary-width operations and aligned memory allocators. Build system (CMake) supports cross-platform compilation with configurable SIMD modes and optional Snappy integration. The library is validated through extensive benchmarks, unit tests, and real-world usage in systems like DuckDB and Apache Lucene.
This repository is the official reference implementation of the VP8 and VP9 video codec standards (libvpx), maintained by the WebM project. It provides a C library for encoding and decoding VP8/VP9 video streams, with support for multiple container formats (IVF, WebM, Y4M), real-time and two-pass encoding, scalable video coding (SVC), and high-bit-depth profiles (10/12-bit). The architecture is a classic monolithic codec library: a public API layer (`vpx_codec_*` functions) dispatches to internal encoder/decoder state machines, which in turn call a large set of DSP primitives (transforms, quantization, motion estimation, loop filtering, entropy coding) implemented in both portable C and heavily optimized SIMD for x86 (SSE2/SSE4/AVX2/AVX512), ARM (NEON, SVE), MIPS (MSA, DSPR2), PowerPC (VSX), and LoongArch (LSX). The build system uses custom `.mk` files with runtime CPU detection (RTCD) to select the appropriate function pointer at runtime, enabling a single binary to run across diverse hardware. The primary data flow is frame-by-frame: raw YUV frames enter the encoder, are transformed, quantized, entropy-coded, and output as a bitstream; the decoder reverses the process. Rate control models buffer fullness and scene complexity to meet target bitrates. The codebase includes extensive Google Test-based unit tests covering algorithm correctness, bitstream conformance, and performance benchmarks. Example applications (`vpxenc`, `vpxdec`, `simple_encoder`, `simple_decoder`) demonstrate the API and serve as integration tests. Key entry points include `vpx_codec_enc_init()`, `vpx_codec_encode()`, `vpx_codec_dec_init()`, `vpx_codec_decode()`, and the control interface (`vpx_codec_control_`). External dependencies are minimal (only standard C library and optional platform headers); third-party code includes Google Test, libwebm (WebM mux/demux), libyuv (color conversion, bundled), and x86inc.asm (NASM macros). The repository also contains research-oriented tools under `tools/3D-Reconstruction/` (Python/Processing sketches for motion estimation and 3D reconstruction) which are experimental and not part of the core library.
ztk is a CLI proxy that reduces LLM token consumption by intercepting and compressing shell command output. It sits between AI agents (Claude Code, Cursor, Gemini CLI) and the shell, executing commands normally but returning filtered output through a six-stage pipeline. The tool removes noise like diff metadata, test runner verbosity, and redundant directory listings while preserving essential information such as error messages and exit codes. Built in Zig with zero dependencies, it features a Thompson NFA regex engine for runtime filters, SIMD-accelerated text processing, and an mmap'd session cache to avoid repeated output for unchanged commands.
minimp3 is a minimal, high-performance, ISO-conformant MP3 decoder implemented as a single-header C library (minimp3.h). It provides a low-level frame decoding API and a higher-level streaming/seek API (minimp3_ex.h) with callback-based I/O abstraction. The decoder supports SSE and NEON SIMD acceleration, and compiles to a ~30kB binary with zero external dependencies. The repository includes a comprehensive test suite with over 50 test vectors covering standard and edge-case MPEG audio streams, a fuzzing harness, and a full-featured GUI player (using SDL2 and Nuklear) that demonstrates real-world usage. An additional tool (huffopt) generates optimized Huffman tables for Layer III. The library is designed for embedded systems, game engines, and any environment requiring a lightweight MP3 decoder.
This repository contains the example code and homework assignments for Casey Muratori's "Performance Aware Programming" course (computerenhance.com). It is primarily educational material, not a production application. The core deliverable is a fully-functional 8086 instruction decoder, disassembler, and cycle estimator (sim86) implemented in C++, with multi-language bindings (C#, Python, Rust, Go, Node.js, WebAssembly, etc.). The rest of the repository comprises over 150 standalone C++ programs and x86 assembly listings that systematically explore performance measurement, memory hierarchy, SIMD (including AVX and FMA), floating-point math optimization (haversine distance), and profiling techniques. Each listing is self-contained, often including a custom repetition tester for accurate benchmarking. The architecture is a collection of independent executables that share common infrastructure via file inclusion (buffer, platform metrics, JSON parser, profiler macros). There is no build system beyond Windows batch scripts; portability to Linux/macOS is community-contributed and not officially tested. The code is well-crafted for teaching low-level performance concepts but is not designed for production reuse, though several components (the 8086 decoder, repetition tester, dead-code prevention macros) could be extracted with minimal effort.