thehardwarestore.ai

MediaPipe is a cross-platform framework for building multimodal machine learning pipelines, primarily targeting on-device inference. It provides a graph-based execution model where modular 'calculators' process typed 'packets' through directed acyclic graphs. The framework includes a rich library of pre-built calculators for image processing, tensor manipulation, audio analysis, and ML inference, as well as high-level task APIs for vision (face/hand/pose landmarking, image classification, segmentation), text (classification, language detection), audio (classification), and generative AI (LLM inference). The architecture is layered: a core C++ framework (mediapipe/framework/) defines the graph runtime, stream handlers, and packet infrastructure; a calculator library (mediapipe/calculators/) provides reusable processing nodes; and task-specific APIs (mediapipe/tasks/) offer consistent interfaces across C++, Java, Python, iOS, Web, and C. The system relies on TensorFlow Lite (LiteRT) for model execution, OpenCV for image operations, protobuf for configuration, and Abseil for utilities. Key entry points include the Python package (mediapipe/__init__.py), the C++ task APIs (mediapipe/tasks/cc/), and the web TypeScript modules (mediapipe/web/). The codebase is large and actively developed, with ongoing migration from older API patterns to newer ones (api2/api3), and some deprecated components (legacy Python solutions, older calculator interfaces).

Moonshine Voice is an open-source AI toolkit for building real-time voice agents and applications with on-device speech-to-text, text-to-speech, and voice cloning capabilities. The system emphasizes privacy, low latency, and offline operation through a cross-platform C++ core with language bindings for Python, Swift, Android, and WebAssembly. It includes optimized models for various platforms (from microcontrollers to desktops), multilingual grapheme-to-phoneme conversion, speaker diarization, and embedding-based phrase matching. The architecture features modular components for audio processing, model inference, and language-specific rules, with examples demonstrating microphone transcription, streaming TTS, agent workflows, and embedded deployments on RP2350 microcontrollers.

Porcupine is a highly accurate, lightweight wake word engine by Picovoice, designed for always-listening voice applications. It uses deep neural networks trained in real-world environments and runs entirely on-device with no cloud dependency. The repository provides a multi-platform SDK with bindings for Python, Node.js, Java, .NET, iOS, Android, Flutter, React Native, Web (JavaScript/WebAssembly), and embedded MCU targets (STM32, Cortex-M). Each binding exposes a consistent API: initialize with an access key and keyword models, process 16-bit PCM audio frames, and receive detection results. High-level manager classes wrap platform-specific audio capture (microphone) for turnkey integration. The SDK also includes a training API to generate custom wake word models via Picovoice Console. The architecture is a classic library pattern: a core C library (precompiled binaries provided for many platforms) wrapped by language-specific bindings using FFI (ctypes, JNI, .NET FFI, WebAssembly). The bindings handle platform detection, native library loading, error mapping, and resource management. Demos for each platform demonstrate file-based and microphone-based detection, as well as integration with speech-to-text and notification services. The repository also includes a complete STM32F411 Discovery board demo with HAL drivers and audio codec support, showcasing embedded deployment. Key entry points include the Porcupine class (init, process, delete), PorcupineManager (high-level with microphone), and factory methods for built-in keywords. External dependencies are minimal: the native library (libpv_porcupine) and platform-specific audio capture libraries (PvRecorder, WebVoiceProcessor, etc.). The SDK requires a Picovoice AccessKey for authentication. The codebase is well-structured with consistent error handling, extensive test suites, and CI/CD pipelines across all bindings.

VieNeu-TTS is an advanced on-device Vietnamese text-to-speech library with instant voice cloning, supporting bilingual (Vietnamese/English) synthesis. The system provides a Python SDK (`vieneu`) with multiple backends: v3 Turbo (default, 48kHz, ONNX or PyTorch), v3 Nano (lightweight 24kHz, ONNX only), and legacy v1/v2 (deprecated). It includes a Gradio web UI (`vieneu-web`), a FastAPI streaming server, and a browser-based HTML/JS client. The core architecture is a factory pattern (`Vieneu(mode=...)`) that returns the appropriate engine, all inheriting from a shared base class. Inference can run entirely torch-free on CPU via ONNX Runtime, with optional GPU acceleration (CUDA) for batched generation and lower latency. Feature highlights include instant voice cloning from a 3–8 second reference clip using a speaker encoder (ONNX) and denoiser, inline emotion cues (`[cười]`, `[thở dài]`), frame-level streaming, SRT subtitle-to-speech, and LoRA fine-tuning for custom voices (requires ~6GB GPU). The system is deployed via Docker with CPU and GPU profiles, and has CI/CD pipelines for testing and PyPI publishing. The codebase is well-structured with clear separation of concerns, though it carries some legacy weight from older model versions and a few deprecated parameters. Other notable components include a sliding-window repetition penalty, a babble guard heuristic for short utterances, and a voice persistence system for user-defined presets.