ncnn is a high-performance neural network inference framework designed for mobile, embedded, and desktop deployment. It provides a self-contained C++ library with no mandatory external dependencies, supporting CPU inference with extensive SIMD optimizations (ARM NEON, x86 SSE/AVX/AVX512, RISC-V Vector, MIPS MSA, LoongArch LSX/LASX) and GPU inference via Vulkan. The framework includes a comprehensive set of neural network layers (convolution, pooling, normalization, activation, recurrent, attention, etc.) and supports multiple data types (float32, float16, bfloat16, int8). The companion pnnx tool converts models from PyTorch (TorchScript) and ONNX into ncnn format using a sophisticated graph rewriting pipeline. Python bindings and a model zoo are provided for rapid prototyping. The codebase is production-grade, used in Tencent applications like QQ and WeChat, and is actively maintained with support for modern architectures and operations (e.g., scaled dot-product attention, rotary embeddings).
Upscayl is a cross-platform desktop application for AI-powered image upscaling, built with Electron and Next.js. It uses Real-ESRGAN models via the ncnn inference engine (Vulkan backend) to upscale images by 2x-4x. The application provides a rich UI with model selection, batch processing, double upscaling, custom model support, and extensive settings (GPU selection, tile size, TTA mode, metadata preservation). It also includes a cloud API (Upscayl Cloud) for server-side upscaling, documented via OpenAPI. The codebase is well-structured with clear separation between Electron main process and Next.js renderer, using Jotai for state management, Radix UI primitives, and a comprehensive localization system supporting 17 languages. While primarily an application, it contains pre-trained neural network model files (.param, .bin) that are research artifacts, giving it a hybrid character.
This repository is a mobile SDK for real-time interactive AI avatars, primarily targeting Android and iOS platforms. It provides on-device inference for speech recognition (via WeNet) and face/body segmentation (via ncnn-based U-Net models), enabling low-latency lip-sync and avatar animation driven by audio input. The SDK is structured as a C++ core with platform-specific wrappers (Java for Android, Objective-C for iOS), and includes bundled third-party libraries (ncnn, OpenCV, FFmpeg, ONNX Runtime) for computer vision, audio processing, and neural network inference. In production, the system ingests 16kHz PCM audio from a microphone, extracts MFCC features, runs streaming speech recognition, and simultaneously processes the audio through a U-Net model to generate facial animation frames. These frames are rendered using OpenGL ES (Android) or Metal (iOS) with alpha blending for compositing onto a background. The SDK also includes model download, encryption (AES-128-CBC), and resource management utilities. The architecture is monolithic: all core logic lives in a single C++ module (duix-sdk), with platform-specific integration code in the app directories. The demo app in Android provides a fully functional test harness. While the SDK is feature-complete and demonstrates significant engineering effort in mobile AI inference, the codebase exhibits several quality issues: duplicated third-party headers across architectures, hardcoded encryption keys, outdated library versions (OpenCV 4.6.0, ncnn 2022/2023), and minimal inline documentation. The high star count and fork activity suggest a strong community, but the code itself is functional rather than exemplar. The most reusable components are the reference-counted memory allocator (dh_mem.h) and the AES stream decryption wrapper (gaes_stream.h), which can be extracted for general-purpose use.
Flowframes is a Windows desktop application for AI-based video frame interpolation. It provides a GUI (WinForms) to orchestrate interpolation using models like RIFE, DAIN, and FLAVR, running on CUDA (PyTorch) or NCNN (Vulkan) backends. The core pipeline extracts frames from input videos via FFmpeg, runs AI inference to generate intermediate frames, and re-encodes the result. It supports batch processing, resume from interruption, various output formats (video, GIF, image sequences), and hardware acceleration detection. The architecture is a monolithic C# application that spawns external processes (FFmpeg, Python scripts, NCNN binaries) and manages their lifecycle. The codebase includes a legacy CodeLegacy folder with near-duplicate files, indicating incomplete refactoring. It also bundles a large vendored Python environment (pip, setuptools, etc.) for the AI runtimes, which introduces maintenance overhead. Despite these issues, the application is functional and actively maintained, with useful utilities like a Fraction struct for rational FPS arithmetic and a rolling average class for performance monitoring.