$ show mybigday/llama.rn
This repository is a high-performance React Native library (llama.rn) that enables on-device Large Language Model (LLM) inference on iOS and Android devices. It wraps the C++ llama.cpp core using JavaScript Interface (JSI) and TurboModules, allowing mobile applications to run GGUF-formatted models locally without cloud dependencies. The architecture consists of a TypeScript/JavaScript layer for the React Native interface, a C++ core for inference and model loading, and native modules for Android (Java/Kotlin) and iOS (Objective-C++). Key capabilities include hardware acceleration via Apple Metal and Qualcomm Hexagon DSP, multimodal support (vision/audio), parallel decoding, and structured output generation. The project includes a comprehensive example app demonstrating chat interfaces, model management, and benchmarking tools.
The LLM identified 5 files worth highlighting for reuse:
An Expo Config Plugin that automates the configuration of native build settings (C++20, entitlements, libraries) required for llama.rn. It demonstrates a reusable pattern for integrating native mobile dependencies into Expo projects without ejecting, handling platform-specific build flags and environment variables programmatically.
Implements a template-based ContextManager for tracking native context pointers and managing the lifecycle of JSI contexts. It prevents race conditions during context release and ensures safe memory management between the JavaScript runtime and native C++ code, a critical pattern for high-performance mobile bindings.
Manages parallel decoding requests using a slot-based architecture. It implements a producer-consumer pattern for handling concurrent inference requests with state isolation, allowing multiple users or tasks to run LLM inference simultaneously without blocking. This pattern is reusable for any high-throughput inference engine.
Automates the download and verification of pre-built native binaries (xcframework, .so) during npm install. It includes SHA-256 verification for security and handles platform-specific extraction, providing a robust pattern for distributing native mobile libraries with minimal user configuration.
Provides streaming-aware UTF-8 parsing utilities used throughout the inference engine. It handles character boundary detection and encoding validation efficiently, a reusable component for any text processing pipeline that needs to handle variable-length encodings without buffering entire strings.