GenieX is a production-grade, on-device generative AI inference SDK and CLI developed by Qualcomm for Snapdragon platforms. It provides a C core with a plugin architecture supporting two inference backends: llama.cpp (GGUF models on CPU/GPU/NPU) and QAIRT (pre-compiled bundles on Hexagon NPU). The system includes a Rust-based model manager that downloads, caches, and resolves models from Hugging Face, Qualcomm AI Hub, Docker Hub, and local filesystem, with resumable chunked downloads and progress reporting. It exposes multiple client interfaces: a Go CLI with subcommands for inference, an OpenAI-compatible HTTP server (streaming chat/completions, tool calls), Python bindings with high-level AutoModel factories, Android bindings via JNI/Kotlin, and Go bindings. The architecture is cleanly layered: C SDK → plugin backends → hardware, with a separate model manager process. The codebase demonstrates strong engineering practices: extensive test suites (unit, integration, device-level), CI/CD pipelines, versioned releases, and cross-platform support (Linux ARM64, Windows ARM64, Android). While primarily an application, it includes benchmark infrastructure and quality evaluation tools that could be used for research purposes.
This repository is a high-performance React Native library (llama.rn) that enables on-device Large Language Model (LLM) inference on iOS and Android devices. It wraps the C++ llama.cpp core using JavaScript Interface (JSI) and TurboModules, allowing mobile applications to run GGUF-formatted models locally without cloud dependencies. The architecture consists of a TypeScript/JavaScript layer for the React Native interface, a C++ core for inference and model loading, and native modules for Android (Java/Kotlin) and iOS (Objective-C++). Key capabilities include hardware acceleration via Apple Metal and Qualcomm Hexagon DSP, multimodal support (vision/audio), parallel decoding, and structured output generation. The project includes a comprehensive example app demonstrating chat interfaces, model management, and benchmarking tools.
This repository is a comprehensive collection of TensorFlow Lite example applications for on-device machine learning, covering tasks such as image classification, object detection, pose estimation, audio classification, BERT question answering, gesture recognition, and more. It provides complete Android (Kotlin/Java), iOS (Swift), Raspberry Pi, and web implementations, each demonstrating best practices for camera integration, hardware delegate selection, and real-time inference. Additionally, the repository includes the TensorFlow Lite Model Maker library, which offers high-level APIs for training and exporting custom TFLite models from user data, along with research/educational models (DenseNet, pix2pix, NMT). While the codebase shows some duplication across platforms and occasional legacy API usage, it remains a valuable reference for developers building edge ML applications.