$ show ggml-org/whisper.cpp
whisper.cpp is a high-performance C/C++ inference engine for OpenAI's Whisper automatic speech recognition (ASR) model, with additional support for a wide range of large language models (LLMs) via an integrated llama.cpp fork. The core library is dependency-free and uses the ggml tensor library to run on CPU, GPU (CUDA, Metal, Vulkan, OpenCL, SYCL, etc.), and specialized hardware (Apple Neural Engine, Hexagon DSP, etc.). It provides a clean C API, extensive language bindings (Go, Java, Ruby, JavaScript, Swift, etc.), and a rich set of example applications including a CLI transcriber, real-time microphone streaming, an HTTP server with OpenAI-compatible API, voice-controlled chess, and a talk-llama chatbot. The repository also includes model conversion scripts (PyTorch to ggml, CoreML, OpenVINO), quantization tools, and evaluation benchmarks (LibriSpeech, Earnings21). While primarily an application library, it contains research-oriented features like grammar-constrained decoding, beam search, and experimental model architectures. The codebase is actively maintained with a large community, though the sheer number of hardware backends and model variants introduces some maintenance complexity.
The LLM identified 4 files worth highlighting for reuse:
A standalone BNF grammar parser that converts a textual grammar definition into a parse state machine for constrained text generation. It is used by whisper.cpp for voice commands and by the talk-llama example for structured output. The parser is self-contained (only depends on standard library and a small common header) and can be extracted into any text generation project to enforce output format constraints.
Shared audio input/output utilities used by all whisper.cpp examples. Provides functions to read WAV files, convert audio formats, and handle PCM data. It abstracts away platform-specific audio handling (e.g., SDL, miniaudio) and can be reused in any C/C++ project that needs to load audio for ASR or audio processing.
A Python script that automates benchmarking of whisper.cpp by running the CLI with various model sizes, thread counts, and processor configurations. It parses the output timing data and writes CSV results. The script is parameterized and can be adapted to benchmark any CLI tool that outputs structured timing information, making it a reusable pattern for performance regression testing.
A Python script that converts a PyTorch Whisper model to CoreML format with optimizations for Apple Neural Engine (ANE). It replaces standard attention layers with Conv2d-based equivalents and handles KV cache hooks for autoregressive decoding. The script demonstrates a reusable pattern for converting transformer-based ASR models to CoreML with ANE-specific constraints, applicable to other encoder-decoder models.