GenieX is a production-grade, on-device generative AI inference SDK and CLI developed by Qualcomm for Snapdragon platforms. It provides a C core with a plugin architecture supporting two inference backends: llama.cpp (GGUF models on CPU/GPU/NPU) and QAIRT (pre-compiled bundles on Hexagon NPU). The system includes a Rust-based model manager that downloads, caches, and resolves models from Hugging Face, Qualcomm AI Hub, Docker Hub, and local filesystem, with resumable chunked downloads and progress reporting. It exposes multiple client interfaces: a Go CLI with subcommands for inference, an OpenAI-compatible HTTP server (streaming chat/completions, tool calls), Python bindings with high-level AutoModel factories, Android bindings via JNI/Kotlin, and Go bindings. The architecture is cleanly layered: C SDK → plugin backends → hardware, with a separate model manager process. The codebase demonstrates strong engineering practices: extensive test suites (unit, integration, device-level), CI/CD pipelines, versioned releases, and cross-platform support (Linux ARM64, Windows ARM64, Android). While primarily an application, it includes benchmark infrastructure and quality evaluation tools that could be used for research purposes.
Thunderbolt is a full-stack, cross-platform AI chat and agent client built by Mozilla Thunderbird. It ships as a React/Vite web app, a Tauri desktop shell (macOS/Windows/Linux), native iOS/Android apps, and a Bun-based CLI. The system provides chat with multiple LLM providers (OpenAI-compatible, Anthropic, custom Tinfoil enclave), a skills system for user-defined slash-command instructions, voice mode with STT/TTS, MCP tool integration, ACP (Agent Client Protocol) remote agent support, and a widget system for rich chat outputs (weather, maps, citations, document previews, link previews). The backend is an Elysia/Bun server handling OIDC/SAML/OTP auth, an inference usage ledger with signed receipts, a CORS-bypassing proxy with streaming, and PowerSync sync coordination. Data flows from local SQLite (wa-sqlite/bun-sqlite) through an E2E-encrypted PowerSync pipeline to PostgreSQL, with device registration, envelope key storage, and canary proof-of-possession for recovery. The CLI provides a separate agent harness with iroh P2P transport, device-grant auth, and a workspace jail for safe file operations. The repo also contains a substantial AI evaluation framework (scenario-based scoring with Wilson confidence intervals), a Pulumi/K8s/Docker deployment stack, and a marketing/docs site in Astro. While the dominant character is clearly an application, it has a notable data-engineering flavor in its inference usage tracking, eval harness, and telemetry pipelines.
Swarm is a production-grade Swift-native agent runtime framework for building multi-agent conversational systems with type-safe tools, on-device Apple Foundation Models, composable workflows, memory, guardrails, and observability. The architecture is modular: core agent execution (Agent, Workflow, Job) with pluggable inference providers (Foundation Models, OpenAI-compatible, mock), memory backends (conversation, sliding window, summary, vector, persistent via SwiftData or Wax SQLite), and tool integration (built-in tools, MCP protocol, web search). Key entry points include Agent.run()/stream(), Workflow.execute(), and the SwarmMCPServerService for exposing tools via MCP. Data flows involve agent turns with tool calls, memory retrieval/consolidation, and durable workflow checkpointing via the HiveCore deterministic runtime. External dependencies include Apple FoundationModels (macOS 26+), swift-log, swift-syntax (for macros), and optional Wax (SQLite vector store) and OpenTelemetry SDK. The framework emphasizes compile-time safety via @Tool and @Agent macros, and provides resilience patterns (retry, fallback, circuit breaker, rate limiting) and observability (tracing, metrics, OpenTelemetry).
Open Dungeon is a fully local AI roleplay application that generates both narrative text and inline scene images on-device. Built as a Next.js 16 application with a React 19 frontend, it provides a rich interactive fiction experience where users can write actions, dialogue, or narration, and the AI narrator responds with streaming prose. The system supports multiple text backends: local Ollama models (Gemma 4 QAT variants) or any OpenAI-compatible server (LM Studio, llama.cpp, OpenRouter, etc.). Image generation is handled by a separate Python HTTP server that wraps optimized FLUX.2-klein backends (MFLUX/MLX on Apple Silicon, PyTorch SDNQ on CUDA/CPU) or an existing ComfyUI instance. The architecture is a monolith with a Next.js server handling API routes, SQLite for persistence, and an optional sidecar image server. Key entry points include the main chat UI (src/app/page.tsx), the story API (src/app/api/story/route.ts) which handles streaming narration, and the image generation API (src/app/api/images/route.ts). The codebase demonstrates strong engineering practices: thorough streaming support with artifact filtering, robust error handling with timeouts and fallbacks, clean database abstraction, and well-structured Python image server with resident model support.
Google AI Edge Gallery is an Android application that serves as a showcase and playground for on-device AI models, primarily Gemma 4 and other LiteRT-compatible LLMs. It provides a rich set of interactive tasks including AI Chat, Prompt Lab, Ask Image, Audio Scribe, Tiny Garden (a mini-game powered by a 270M model), Mobile Actions (device control via function calling), and an Agent Skills system with MCP (Model Context Protocol) integration. The app manages the full lifecycle of on-device models: discovery (via HuggingFace), download, initialization, inference, and benchmarking. It features a plugin architecture via the CustomTask interface, allowing new tasks to be registered through Hilt dependency injection. Skills are defined as markdown instruction files paired with JavaScript functions executed in a WebView, enabling extensible tool use. The codebase is well-structured with Jetpack Compose UI, Proto DataStore persistence, Firebase Analytics, and a clean separation of concerns. It is a professional-grade demo application that demonstrates advanced on-device AI capabilities, though some areas (e.g., skill duplication, large ViewModels) show signs of rapid development.
Local Browser is a Chrome extension that enables on-device AI-powered web automation using WebLLM. The system implements a two-agent architecture (Planner + Navigator) where users enter natural language tasks in the popup UI, which are then executed through a loop of DOM observation, action planning, and browser interaction. All AI processing occurs locally on the device with no cloud APIs or API keys required. The extension features site-specific state machines for common tasks like Amazon shopping and YouTube navigation (minimizing LLM calls), obstacle detection for login requirements/CAPTCHAs/out-of-stock items, and robust action execution with retry logic and semantic selectors. Built with React, TypeScript, Vite, and CRXJS for MV3 Chrome extension architecture.
This is the Qualcomm Innovators Development Kit (QIDK), a collection of Android sample applications and tools demonstrating on-device AI inference using the Qualcomm Neural Processing SDK (SNPE) and Qualcomm Neural Network (QNN) runtime. The repository covers computer vision (object detection, pose estimation, super-resolution, image enhancement), NLP (sentiment analysis, question answering, automatic speech recognition), and generative AI (LLM chat, speech-to-image, ASR-LLM-TTS pipeline). Each solution is a standalone Android app with native C++ JNI code that integrates the SNPE/QNN SDK for accelerated inference on Snapdragon platforms. The repository also includes tools like `pysnpe_utils` (Python library for model conversion and execution), `snpe-helper` (Python wrapper for SNPE runtime), and low-level Hexagon compute examples (QHCI/HVX). While the codebase is functional and covers a wide range of use cases, it suffers from significant code duplication across solutions, hardcoded paths, and version coupling to specific SDK releases. The architecture is a collection of loosely coupled sample apps rather than a unified framework, limiting cross-project reuse. The commit history shows ongoing maintenance, but the last commit date appears anomalous (2026), and several files reference future copyright years, suggesting pre-release content.