$ show duixcom/Duix-Mobile
This repository is a mobile SDK for real-time interactive AI avatars, primarily targeting Android and iOS platforms. It provides on-device inference for speech recognition (via WeNet) and face/body segmentation (via ncnn-based U-Net models), enabling low-latency lip-sync and avatar animation driven by audio input. The SDK is structured as a C++ core with platform-specific wrappers (Java for Android, Objective-C for iOS), and includes bundled third-party libraries (ncnn, OpenCV, FFmpeg, ONNX Runtime) for computer vision, audio processing, and neural network inference. In production, the system ingests 16kHz PCM audio from a microphone, extracts MFCC features, runs streaming speech recognition, and simultaneously processes the audio through a U-Net model to generate facial animation frames. These frames are rendered using OpenGL ES (Android) or Metal (iOS) with alpha blending for compositing onto a background. The SDK also includes model download, encryption (AES-128-CBC), and resource management utilities. The architecture is monolithic: all core logic lives in a single C++ module (duix-sdk), with platform-specific integration code in the app directories. The demo app in Android provides a fully functional test harness. While the SDK is feature-complete and demonstrates significant engineering effort in mobile AI inference, the codebase exhibits several quality issues: duplicated third-party headers across architectures, hardcoded encryption keys, outdated library versions (OpenCV 4.6.0, ncnn 2022/2023), and minimal inline documentation. The high star count and fork activity suggest a strong community, but the code itself is functional rather than exemplar. The most reusable components are the reference-counted memory allocator (dh_mem.h) and the AES stream decryption wrapper (gaes_stream.h), which can be extracted for general-purpose use.
The LLM identified 2 files worth highlighting for reuse:
Provides a simple reference-counted memory allocator (dhmem_alloc, dhmem_ref, dhmem_deref) with a destructor callback. It is a self-contained C header that can be extracted into any C/C++ project requiring manual memory management with automatic cleanup. The implementation uses atomic operations for thread safety and supports aligned allocation. It is a reusable pattern for RAII-like resource management in C.
Implements a custom std::streambuf (GaesIStream) that transparently decrypts AES-128-CBC encrypted data on-the-fly. It wraps a file stream and a supplied AES key, performing block decryption during reads. This is a reusable pattern for decrypting model files or other assets without loading the entire file into memory. The class is self-contained and depends only on the project's AES implementation (aes.h), which can be replaced with any compatible library.