$ show Tencent/ncnn
ncnn is a high-performance neural network inference framework designed for mobile, embedded, and desktop deployment. It provides a self-contained C++ library with no mandatory external dependencies, supporting CPU inference with extensive SIMD optimizations (ARM NEON, x86 SSE/AVX/AVX512, RISC-V Vector, MIPS MSA, LoongArch LSX/LASX) and GPU inference via Vulkan. The framework includes a comprehensive set of neural network layers (convolution, pooling, normalization, activation, recurrent, attention, etc.) and supports multiple data types (float32, float16, bfloat16, int8). The companion pnnx tool converts models from PyTorch (TorchScript) and ONNX into ncnn format using a sophisticated graph rewriting pipeline. Python bindings and a model zoo are provided for rapid prototyping. The codebase is production-grade, used in Tencent applications like QQ and WeChat, and is actively maintained with support for modern architectures and operations (e.g., scaled dot-product attention, rotary embeddings).
The LLM identified 3 files worth highlighting for reuse:
Provides portable SSE implementations of exp, log, sin, cos, pow, and other transcendental functions using Intel SSE intrinsics. This header is self-contained and can be extracted into any C++ project requiring fast vectorized math on x86 platforms. It demonstrates expert-level SIMD optimization and careful handling of edge cases.
ARM NEON counterpart to sse_mathfun.h, implementing vectorized exp, log, sin, cos, and tanh using NEON intrinsics. It is a standalone, reusable component for any ARM-based project needing high-performance math functions. The code is well-structured and includes both float32 and float16 variants.
Defines a unified activation function interface that can be applied to tensors after operations like convolution or inner product. It supports ReLU, sigmoid, tanh, GELU, and other activations via a simple enum and a single function call. This header is small, self-contained, and can be reused in any inference engine or numerical library to avoid code duplication.