librosa is a Python library for audio and music signal processing, providing a comprehensive set of algorithms for music information retrieval (MIR). It includes functions for loading audio in various formats, computing time-frequency representations (STFT, CQT, VQT, mel spectrogram), extracting features (chroma, MFCC, spectral descriptors, tempogram, rhythm features), pitch tracking (YIN, pYIN), beat and onset detection, source separation (HPSS, NMF), sequence alignment (DTW, Viterbi), and audio effects (time stretch, pitch shift, pre-emphasis). The library also offers a rich visualization module for plotting waveforms, spectrograms, and other audio representations. The architecture is modular, organized into subpackages: `librosa.core` for core DSP and I/O, `librosa.feature` for feature extraction, `librosa.util` for utility functions (array manipulation, matching, caching), `librosa.filters` for filter bank construction, `librosa.display` for visualization, and domain-specific modules like `librosa.beat`, `librosa.onset`, `librosa.decompose`, `librosa.effects`, `librosa.segment`, and `librosa.sequence`. Submodules are lazily loaded to reduce import time. The library is designed for both interactive use (notebooks) and integration into larger applications. Key entry points include `librosa.load` for audio I/O, `librosa.stft`/`librosa.cqt` for time-frequency transforms, `librosa.feature.*` for feature extraction, `librosa.beat.beat_track` for beat tracking, `librosa.onset.onset_detect` for onset detection, `librosa.display.specshow`/`librosa.display.waveshow` for visualization, and `librosa.effects.*` for audio effects. Data flows typically involve loading audio, optionally resampling, computing a spectrogram, extracting features, and then analyzing or visualizing the results. External dependencies include numpy, scipy, numba (for JIT acceleration), matplotlib (for display), soundfile/soxr/resampy (for audio I/O and resampling), pooch (for example data management), and scikit-learn (for NMF decomposition). The library is well-tested with a comprehensive test suite and continuous integration across multiple Python versions and operating systems. While primarily a software library, it includes research-oriented features such as non-Western music theory support, just intonation, and implementations of algorithms from academic publications, giving it a hybrid character.
FFmpeg is a comprehensive, production-grade multimedia framework providing libraries and command-line tools for encoding, decoding, transcoding, muxing, demuxing, filtering, streaming, and playing audio, video, subtitles, and related metadata. The codebase is organized into several core libraries: libavcodec (codec implementations), libavformat (container formats), libavfilter (audio/video filters), libavutil (utilities), libavdevice (device I/O), libswresample (audio resampling), and libswscale (image scaling/colorspace conversion). The primary user-facing tools are ffmpeg (conversion/streaming), ffplay (media player), and ffprobe (media analysis). The architecture is modular, with each library exposing a public API and internally using a plugin-like registration system for codecs, demuxers, muxers, filters, and bitstream filters. Hardware acceleration is supported via Vulkan, CUDA, VA-API, D3D11VA, D3D12VA, VideoToolbox, MediaCodec, and others, with architecture-specific SIMD optimizations for x86, ARM, RISC-V, MIPS, LoongArch, and PowerPC. The project is actively maintained with extensive testing infrastructure including fuzzers, unit tests, and a checkasm framework for verifying assembly optimizations.
Auto-editor is a command-line application for automatically editing video and audio content. It analyzes media files to detect silent or still sections (based on audio loudness, motion, black frames, or subtitles) and automatically cuts them out, producing a more concise output. The tool supports a wide range of input and output formats, leveraging FFmpeg for media processing and offering export options tailored to popular video editing software like Adobe Premiere Pro, Final Cut Pro, DaVinci Resolve, Shotcut, and Kdenlive. The architecture is modular, with distinct modules for analysis (audio, motion, black detection, subtitle processing), rendering, export format handling, and command-line interface. Key features include a plugin-like system for edit methods (e.g., audio-based, motion-based), a caching mechanism to avoid redundant analysis, support for keyboard input via microphone, and integration with Whisper.cpp for speech transcription. The project also provides a web interface and supports advanced features like transitions, speed ramping, volume ducking, and keyframe-based effects. The codebase demonstrates strong engineering practices through its use of a custom expression language for defining edit decisions, SIMD-optimized performance critical paths, and a clear separation between media processing logic and user-facing functionality. Despite its complexity and reliance on external multimedia libraries, the project maintains active development with regular updates and cross-platform support.
The KFR library is a high-performance C++ framework for digital signal processing (DSP) and audio applications. It provides optimized implementations of FFT/DFT, filter design (FIR/IIR), sample rate conversion, audio file I/O (supporting WAV, FLAC, MP3, AIFF, CAF, etc.), SIMD-accelerated mathematical operations, and tensor/multi-dimensional array operations. The library is designed for real-time processing and scientific computing, with architecture-specific optimizations for x86 SIMD, ARM NEON, and RISC-V RVV, and runtime dispatch for multiarchitecture support. Key architectural features include a header-only design (with optional separate compilation), expression templates for lazy evaluation and operator fusion, a portable SIMD abstraction layer, and a tensor system with NumPy interoperability. The library is used in production by scientific collaborations such as LIGO, Virgo, and KAGRA for gravitational-wave signal processing, benchmarked against FFTW. It employs a dual licensing model (GPLv2+/commercial) and requires C++20 for compilation. The codebase demonstrates strong engineering practices through reusable components such as the SIMD vector abstraction (vec<T, N>), expression template system, tensor class, and audio data handling unit. These components are self-contained, exhibit non-obvious skill in C++ template metaprogramming, and can be extracted for use in other projects with minimal modification. The library is actively maintained, with recent commits and CI/CD pipelines, and shows low technical drift by keeping pace with modern C++ standards and hardware advancements.
MediaPipe is a cross-platform framework for building multimodal machine learning pipelines, primarily targeting on-device inference. It provides a graph-based execution model where modular 'calculators' process typed 'packets' through directed acyclic graphs. The framework includes a rich library of pre-built calculators for image processing, tensor manipulation, audio analysis, and ML inference, as well as high-level task APIs for vision (face/hand/pose landmarking, image classification, segmentation), text (classification, language detection), audio (classification), and generative AI (LLM inference). The architecture is layered: a core C++ framework (mediapipe/framework/) defines the graph runtime, stream handlers, and packet infrastructure; a calculator library (mediapipe/calculators/) provides reusable processing nodes; and task-specific APIs (mediapipe/tasks/) offer consistent interfaces across C++, Java, Python, iOS, Web, and C. The system relies on TensorFlow Lite (LiteRT) for model execution, OpenCV for image operations, protobuf for configuration, and Abseil for utilities. Key entry points include the Python package (mediapipe/__init__.py), the C++ task APIs (mediapipe/tasks/cc/), and the web TypeScript modules (mediapipe/web/). The codebase is large and actively developed, with ongoing migration from older API patterns to newer ones (api2/api3), and some deprecated components (legacy Python solutions, older calculator interfaces).
FDSoundActivatedRecorder is a Swift library for iOS that implements sound-activated audio recording. It automatically starts recording when sound exceeds a dynamic threshold and stops when silence is detected, saving the output as an m4a file with fade-in/fade-out to prevent clicks. The core logic uses a moving average of audio power levels to dynamically adjust trigger thresholds, making it robust to varying noise environments. Built with modern Swift concurrency (actor) for thread safety, it provides a delegate protocol for callbacks and metering updates. The library is self-contained, depends only on standard iOS frameworks, and includes an example app demonstrating integration via SwiftUI.
FluidVoice is a macOS desktop application for real-time voice dictation, meeting transcription, and AI-powered text enhancement. It provides a local-first dictation experience with on-device speech recognition (Parakeet, Nemotron, Whisper, Apple Speech) and optional cloud AI post-processing. The application features a command mode for AI chat, meeting recording with speaker diarization and acoustic echo cancellation, and a local HTTP API for external integrations. The architecture is a native SwiftUI/AppKit application with a service-oriented design. Audio capture uses CoreAudio and ScreenCaptureKit, with a vendored WebRTC audio processing stack for echo cancellation (AEC3), noise suppression, and voice activity detection. Speech recognition is handled by multiple providers, with model downloads from Hugging Face. The application includes a durable PCM storage system with chunk ledger for meeting audio, a signed model release feed for secure updates, and a comprehensive settings/persistence layer using SQLite and UserDefaults. Key entry points include FluidApp.swift (main application), LocalAPIRouter.swift (HTTP API server), MeetingSummaryProvider.swift (AI meeting summaries), ASRService.swift (speech recognition orchestration), and DictationAppSession.swift (dictation workflow). External dependencies include macOS system APIs, SQLite, vendored WebRTC audio processing, and various AI provider APIs (OpenAI, Anthropic, etc.). The repository also contains research-grade audio processing code (WebRTC AEC3, AGC2, RNN VAD) and pronunciation learning evaluation tools, but the dominant character is a production macOS application with extensive testing and CI/CD.
CamillaDSP is a real-time audio stream processing engine written in Rust. It captures audio from a configurable backend (ALSA, PulseAudio, PipeWire, CoreAudio, WASAPI, ASIO, or file), processes it through a user-defined pipeline of filters, mixers, compressors, and other DSP components, and outputs it to a playback device. The pipeline is defined via YAML configuration, supporting biquad filters, FIR convolution (with hand-optimized NEON and AVX+FMA SIMD kernels), loudness, limiting, noise gates, and dithering. A websocket-based control interface allows runtime adjustments of volume, mute, faders, and config reloads. The architecture is modular with separate threads for capture, processing, and playback, using a PI rate controller for buffer level regulation. Benchmarks demonstrate attention to performance. While primarily an application, the DSP filter implementations and utility components (ring buffer, conversion routines, rate controller) are reusable building blocks for audio software.
Whisper is a speech recognition system that provides both a command-line interface and a Python library for transcribing and translating audio. The system processes audio by converting it to log-Mel spectrograms, then uses a Transformer-based encoder-decoder model to predict text tokens. Key components include audio processing, model architecture, decoding strategies, tokenization, and utilities for word-level timestamp alignment. The codebase is modular, well-tested, and actively maintained with recent updates supporting modern Python and PyTorch versions.
Silero VAD is a pre-trained enterprise-grade Voice Activity Detector library that provides speech detection capabilities for audio signals. The system consists of a core PyTorch model (also available in ONNX format) with Python utilities for audio processing, speech timestamp extraction, and streaming applications. It supports 8kHz and 16kHz sampling rates and includes a VADIterator class for real-time processing. The repository offers multi-language examples (C++, C#, Go, Haskell, Java, Rust) demonstrating integration, along with tools for model tuning and sequence-based ONNX optimization for improved performance on long audio files.
LedFx is a networked LED light effect controller that synchronizes multiple addressable LED devices (e.g., WLED, LIFX, Nanoleaf, Art-Net, DDP) to real-time audio analysis. It runs as a Python asyncio application with a built-in aiohttp REST API and a prebuilt React frontend served on port 8888. The system captures audio from system loopback, microphone, or network streams (Sendspin), processes it through FFT/melbank/beat detection, and drives audio-reactive effects on virtual LED strips and matrices. It also integrates with now-playing services (MPRIS, SMTC, Sendspin) to extract album art gradients and apply them to effects. The architecture is plugin-based: effects, devices, integrations, and API endpoints are registered via a `BaseRegistry` pattern with voluptuous schema validation. The core runtime (`LedFxCore`) orchestrates device discovery, virtual management, scene/playlist systems, and configuration persistence. The codebase is well-structured with clear separation of concerns, but some modules (e.g., `core.py`, `virtuals.py`) are large and handle multiple responsibilities. The project is actively maintained with comprehensive CI/CD, extensive test coverage (including API integration tests), and detailed documentation for users and developers.
Pyroomacoustics is a Python library for room acoustics simulation and audio array signal processing. It provides a modular framework for creating virtual acoustic environments (rooms with walls, materials, sources, and microphone arrays) and simulating room impulse responses using both image-source and ray-tracing methods, with a C++ backend for performance. Beyond simulation, it offers a comprehensive suite of audio processing algorithms: beamforming (delay-and-sum, MVDR, rake filters), direction-of-arrival estimation (MUSIC, SRP-PHAT, FRIDA, CSSM, TOPS, WAVES), blind source separation (AuxIVA, ILRMA, FastMNMF, TRINICON), adaptive filtering (NLMS, RLS, subband LMS), and noise reduction (spectral subtraction, subspace, iterative Wiener). It also includes directivity modeling (analytic patterns and measured SOFA data), STFT/DFT transforms with multiple backends, and utility modules for resampling, filterbanks, and metrics (RT60, PESQ). The library is designed for research prototyping and reproducible experiments, with extensive unit tests and CI/CD for multi-platform builds.
SoundTouchJS is a TypeScript audio processing library for real-time time-stretching and pitch-shifting in web applications using the Web Audio API. It implements the WSOLA algorithm and offers alternatives like phase vocoder for extreme ratios, with formant correction for vocal preservation. The monorepo structure includes core processing engine, AudioWorklet implementations, and interchangeable interpolation strategies. Key features include runtime parameter updates via message passing, offline processing capabilities, and observability metrics. The library is designed for integration via AudioWorklet to offload processing from the main thread, with a demo application and Storybook showcasing usage.
SpotiFLAC-Mobile is a cross-platform mobile application (Android and iOS) that enables users to download and play high-fidelity music in lossless formats such as FLAC and DSD. It operates through an extension-based architecture where community-provided plugins supply download sources, metadata, lyrics, and cover art. The application integrates with streaming services like Deezer, Tidal, Qobuz, and Spotify via these extensions, handling authentication, rate limiting, and content resolution. The system is built on a multi-language stack: a core Rust backend handles audio processing, metadata tagging, library management, and network communication; a Flutter (Dart) frontend provides the user interface; and platform-specific code in Kotlin (Android) and Swift (iOS) bridges native features like file system access, USB audio output, and system audio routing. The Rust backend is exposed to mobile languages via UniFFI-generated bindings. Key data flows include: extension-based download orchestration with cancellation and progress reporting, audio file finalization (metadata embedding, ReplayGain scanning, cover art insertion), incremental library scanning with SQLite persistence, and multi-provider lyrics fetching with caching. The application also features hi-res audio authenticity verification using STFT analysis, direct USB audio output for bit-perfect playback, and a comprehensive extension management system with sandboxed JavaScript runtimes. External dependencies include FFmpeg for audio conversion, libusb for USB audio transport, and various web APIs (Deezer, MusicBrainz, LRCLib, Apple Music, etc.) for metadata and lyrics enrichment. The project demonstrates strong engineering practices with extensive unit and integration tests, a modular crate architecture, and reusable patterns for cancellation, progress tracking, and caching.
SpotiFLAC is a cross-platform desktop application (built with Wails v2) that enables users to download high-quality FLAC audio from streaming services (Tidal, Qobuz, Amazon Music, Deezer) by leveraging Spotify metadata. The user inputs a Spotify track, album, playlist, or artist URL; the backend fetches metadata via Spotify's internal GraphQL API using reverse-engineered TOTP authentication, resolves streaming URLs through Songlink/Songstats, then downloads from the chosen service. The app includes a comprehensive suite of audio processing tools: audio analysis (spectrogram, FFT), BPM/key detection (Essentia.js), format conversion, resampling, ReplayGain tagging, lyrics management (LRCLIB), metadata enrichment, and file management. The frontend is built with React 19, TypeScript, Tailwind CSS, and shadcn/ui components, communicating with the Go backend via Wails bindings. The backend handles persistent storage (BoltDB for queue, history, caches), FFmpeg integration, and community API sessions for Tidal/Qobuz/Amazon. The architecture is a monolith desktop app with a clear separation between frontend UI and backend services, but the backend modules are tightly coupled to the app's specific workflow. The app is open-source (MIT) with a Patreon funding model and has a large internationalization footprint (11 languages). Key entry points include the SearchBar for URL input, the download queue for batch processing, and various tool pages (Audio Analysis, Converter, Resampler, ReplayGain, File Manager, Lyrics Manager). External dependencies include Spotify's undocumented API, Tidal/Qobuz/Amazon community APIs, Songlink, LRCLIB, MusicBrainz, FFmpeg, and Essentia.js. The codebase demonstrates significant reverse-engineering effort but carries maintenance risks due to reliance on fragile third-party APIs and hardcoded secrets.
Praat is a cross-platform desktop application for phonetic analysis, speech manipulation, and linguistic research. It provides a rich GUI with interactive editors for viewing and editing sounds, spectrograms, pitch, formants, text annotations, and more. The core architecture is a monolithic C++ application built on a custom object system (Thing/oo_* macros) with automatic serialization, a comprehensive preferences framework, and a scriptable interface. The codebase bundles many external libraries (GSL, eSpeak, FLAC, Opus, LAME, PortAudio, WhisperCPP, etc.) for numerical computing, audio I/O, speech synthesis, and machine learning. It includes modules for linear predictive coding (LPC), cepstral analysis, neural networks (FFNet), optimality theory grammar, multidimensional scaling, and EEG/ERP processing. While the code is well-structured and domain-specific, the heavy reliance on external libraries, custom object system, and platform-specific ifdefs introduce maintenance complexity and moderate technical drift.
Pydub is a mature, high-level Python library for audio manipulation. It provides an immutable `AudioSegment` class that supports slicing, concatenation, overlaying, gain changes, fading, and format conversion using ffmpeg/avconv as a backend for non-WAV formats. The library includes a variety of DSP effects (normalize, speedup, compression, filters, equalization), silence detection/splitting, signal generators (sine, square, sawtooth, white noise), and optional playback via simpleaudio, pyaudio, or ffplay. Its architecture is modular, with core functionality in `audio_segment.py` and effects registered via a decorator pattern. The codebase demonstrates clean abstractions and is well-tested, though it carries some legacy Python 2 compatibility patterns.
This repository implements a Linux-based voice satellite for Home Assistant, acting as a local voice assistant that connects to Home Assistant via the ESPHome protocol (aioesphomeapi). The application runs as a Python asyncio service, handling wake word detection using TensorFlow Lite models (MicroWakeWord and OpenWakeWord), audio input via the soundcard library, audio output via libmpv, and optional WebRTC noise suppression/gain. It also provides a WebSocket-based Peripheral API (port 6055) that allows external hardware controllers (LED rings, buttons, HAT boards) to register entities and receive real-time state events. The system is deployed via Docker with docker-compose and includes example peripheral implementations for devices like the ReSpeaker HATs, Jabra Speak 410, and Satellite1 HAT. The architecture is modular, with separate modules for the ESPHome API server (api_server.py), core voice pipeline protocol (satellite.py), entity management (entity.py), wake word loading (wake_word.py), audio playback (player/libmpv.py), and peripheral integration (peripheral_api.py). The application is configured via environment variables and command-line arguments, with preferences persisted to a JSON file. The codebase includes unit tests for most modules, CI/CD workflows, and a Docker build pipeline. External dependencies include aioesphomeapi, websockets, soundcard, python-mpv, webrtc-noise-gain, pymicro-wakeword, pyopen-wakeword, and zeroconf. Key data flows: Microphone audio is captured, optionally processed by WebRTC for noise suppression and gain, then fed into wake word detection. Upon wake word detection, the voice pipeline sends audio to Home Assistant for speech recognition and intent handling. Responses (TTS, media) are received and played via MPV. The Peripheral API allows external hardware to send button events and receive state changes (e.g., listening, thinking, muted). The system supports multiple wake words, stop word detection, timers, and media playback with playlist support.
qaac is a command-line audio encoder that uses Apple's CoreAudioToolbox (or the reference ALAC encoder) to encode audio to AAC or ALAC formats. It supports a wide range of input formats (WAV, FLAC, WavPack, Opus, Ogg Vorbis, CAF, and more) via runtime-loaded codec libraries, and outputs to MP4/M4A, ADTS, CAF, WAV, or raw PCM. The architecture follows a clean pipeline pattern: input sources implement the ISource interface, pass through optional DSP filters (resampler, normalizer, limiter, compressor, matrix mixer, etc.), and are written to output sinks. Metadata handling is integrated via vendored taglib, and the encoder supports gapless encoding, chapter markers, artwork, and various tagging formats. The codebase is predominantly C++ with platform-specific adaptations for Windows and macOS, and includes several reusable utility components such as a streaming FFT convolver, ring buffer, and cross-platform dynamic library loader.
This is an embedded firmware application for ESP32-based AI voice assistant devices. The code implements a full voice interaction pipeline: microphone input via I2S/PCM, Opus audio encoding/decoding, WebSocket/MQTT communication with a server, wake-word detection (ESP-SR), an LVGL-based user interface with emoji and chat display, an MCP (Model Context Protocol) server exposing device tools to the AI, and support for OTA updates. The architecture is a monolithic FreeRTOS application using ESP-IDF, with a strong board abstraction layer that allows support for over 130 hardware variants (ESP32-S3, C3, C6, P4, etc.) with different audio codecs, displays, cameras, and network modules (WiFi, 4G, Ethernet). The codebase is actively maintained with regular commits and a large community, but suffers from significant code duplication across board definitions, especially in power management and display initialization, which increases maintenance burden.