$ show moonshine-ai/moonshine
Moonshine Voice is an open-source AI toolkit for building real-time voice agents and applications with on-device speech-to-text, text-to-speech, and voice cloning capabilities. The system emphasizes privacy, low latency, and offline operation through a cross-platform C++ core with language bindings for Python, Swift, Android, and WebAssembly. It includes optimized models for various platforms (from microcontrollers to desktops), multilingual grapheme-to-phoneme conversion, speaker diarization, and embedding-based phrase matching. The architecture features modular components for audio processing, model inference, and language-specific rules, with examples demonstrating microphone transcription, streaming TTS, agent workflows, and embedded deployments on RP2350 microcontrollers.
The LLM identified 5 files worth highlighting for reuse:
Implements custom ONNX operators for ZipVoice TTS model to avoid runtime dependencies on external shared libraries. Demonstrates skill in embedding custom ops directly into the ONNX loader, enabling self-contained model deployment across platforms. Reusable for any ONNX-based application requiring custom operations without external .so/.dll files.
Provides a catalog-driven model downloader with progress callbacks, cancellation support, and caching. Solves the common problem of managing large model assets in mobile applications through a self-contained pattern that handles network retries, version checking, and storage management. Extractable to other domains needing reliable asset delivery with user feedback.
High-quality audio resampler supporting multiple algorithms (linear interpolation, polyphase filtering) with quality/speed tradeoffs. Implements efficient sample rate conversion with proper boundary handling and memory management. Demonstrates DSP expertise in designing resampling kernels that minimize artifacts while maintaining low latency. Reusable in any audio processing pipeline requiring sample rate conversion.
Arena-based memory allocator for deterministic memory usage in TensorFlow Lite Micro applications. Provides allocation/deallocation from pre-allocated buffers with alignment support and out-of-memory handling. Critical for microcontroller applications where dynamic allocation is prohibited or unsafe. Demonstrates embedded systems skill in managing memory fragmentation and allocation lifecycles.
Grapheme-to-phoneme conversion framework combining dictionary lookup with rule-based fallback. Features language-specific rule engines via factory pattern and shared infrastructure for IPA tokenization. Demonstrates linguistic processing skill in handling orthographic-to-phonetic conversion across 15+ languages. Reusable as a foundation for TTS or speech recognition systems requiring text-to-phoneme conversion.