$ show Telosnex/fonnx
Fonnx is a Flutter plugin that enables cross-platform, on-device machine learning inference by wrapping the ONNX Runtime engine. It provides a unified Dart API that abstracts away the complexities of native FFI (for iOS, Android, macOS, Windows, Linux) and WebAssembly (for web), allowing developers to run models like OpenAI Whisper, Silero VAD, Magika, and various text embedding models without cloud dependencies. The codebase exhibits high software craftsmanship. Its architecture uses conditional Dart imports to cleanly separate native and web backends, employs Dart isolates for offloading heavy inference tasks, and implements a custom RPC protocol for web workers. A standout feature is its robust native memory management, including a C finalizer tested with sanitizers to prevent leaks during Flutter hot-restarts. While primarily a software library, it bridges the gap between ML research and production by bundling specific, optimized ONNX models and providing a deterministic test corpus with golden outputs. It is an excellent resource for Flutter developers building privacy-focused, offline-first AI applications, though its reliance on pinned native assets and specific build tooling introduces a moderate technical drift risk.
The LLM identified 5 files worth highlighting for reuse:
Core Dart FFI wrapper for the ONNX Runtime C API. Manages session lifecycle, tensor allocation, and inference execution. Implements a robust NativeFinalizer pattern to prevent memory leaks during Flutter hot-restarts, a non-obvious and critical requirement for mobile development. The createSession and tensor helper functions provide a clean, type-safe interface over a complex C library.
A generic, protocol-versioned RPC layer for Dart-Web Worker communication. Handles request/response matching, error propagation, and transferable buffer passing. This pattern cleanly decouples Dart UI code from long-running inference tasks in Web Workers, and is reusable for any Dart web application needing background processing.
A clean, reusable WordPiece tokenizer implementation. Features configurable vocabulary, input caching for performance, and max token limits. Used by multiple embedding models (MiniLM, MSMARCO, MinishLab), demonstrating its abstraction quality. It can be extracted and used in any Dart NLP project requiring standard WordPiece tokenization.
A minimal, high-integrity C module providing safe finalization of ONNX Runtime sessions. Tested with Address Sanitizer (ASan) and Undefined Behavior Sanitizer (UBSan) against 8,192 random states. This component solves the hard problem of reliably cleaning up native resources in a Dart FFI context, demonstrating exceptional attention to correctness and safety.
A Kaldi-compatible streaming filterbank (fbank) feature extractor for audio. Handles frame shifting, windowing, and mel-scale filtering in a streaming fashion. This is a non-trivial DSP component that is critical for real-time keyword spotting and is cleanly separated from the ML model logic, making it a reusable audio frontend.