$ show qualcomm/GenieX
GenieX is a production-grade, on-device generative AI inference SDK and CLI developed by Qualcomm for Snapdragon platforms. It provides a C core with a plugin architecture supporting two inference backends: llama.cpp (GGUF models on CPU/GPU/NPU) and QAIRT (pre-compiled bundles on Hexagon NPU). The system includes a Rust-based model manager that downloads, caches, and resolves models from Hugging Face, Qualcomm AI Hub, Docker Hub, and local filesystem, with resumable chunked downloads and progress reporting. It exposes multiple client interfaces: a Go CLI with subcommands for inference, an OpenAI-compatible HTTP server (streaming chat/completions, tool calls), Python bindings with high-level AutoModel factories, Android bindings via JNI/Kotlin, and Go bindings. The architecture is cleanly layered: C SDK → plugin backends → hardware, with a separate model manager process. The codebase demonstrates strong engineering practices: extensive test suites (unit, integration, device-level), CI/CD pipelines, versioned releases, and cross-platform support (Linux ARM64, Windows ARM64, Android). While primarily an application, it includes benchmark infrastructure and quality evaluation tools that could be used for research purposes.
The LLM identified 4 files worth highlighting for reuse:
A finite state machine that processes token streams to separate reasoning content from final answers. It handles multiple model formats (Gemma, DeepSeek, etc.) by detecting reasoning boundaries. This is a reusable pattern for any application that needs to extract reasoning traces from model output, and it is self-contained with no internal dependencies beyond standard Go.
Implements a parallel, resumable HTTP download executor with a progress bitmap for tracking completed chunks. Supports DEFLATE streaming, scaled progress reporting, and configurable concurrency. This is a generic, reusable component for any project needing robust large-file downloads with resume capability, and it is isolated from business logic.
Parses ZIP64 archives over HTTP using Range requests to read only the central directory, avoiding full download. Supports STORED and DEFLATE methods. This is a highly reusable pattern for any application that needs to inspect or extract files from remote ZIP archives without downloading the entire file, and it is self-contained with a generic transport trait.
Defines the `ModelSource` trait and `BytesSource` enum that abstract over different model sources (HuggingFace, AI Hub, Docker Hub, local filesystem). Each source produces a `Plan` of `FileSpec` with lazy byte references. This is a clean, reusable abstraction for any system that needs to fetch files from multiple heterogeneous backends, and it demonstrates strong trait design.