Ollama is a local LLM serving platform written in Go. It provides a REST API (OpenAI-compatible), CLI, TUI chat, and desktop application for running large language models locally. The system supports multiple inference backends: llama.cpp (CPU/CUDA/ROCm/Vulkan) and MLX (Apple Silicon). It includes a model conversion pipeline from HuggingFace safetensors to GGUF format, GPU hardware discovery, and an agent framework with tool execution (bash, file operations, web search). The architecture is a monolith with subprocess management for inference backends. Key entry points include the CLI (main.go), HTTP server (server/routes.go), runner binary (cmd/runner/main.go), and desktop UI (app/ui/app/). The codebase is actively maintained with extensive test coverage and support for dozens of model architectures. While primarily an application, it has significant data/research components in model conversion, tokenization, and speculative decoding research.
MLX is an array framework for machine learning on Apple Silicon and NVIDIA GPUs, developed by Apple's machine learning research team. It provides a NumPy-like API for Python and C++ with composable function transformations (autograd, vectorization, graph optimization), lazy computation, dynamic graph construction, and multi-backend support (CPU, Apple Metal, NVIDIA CUDA). The framework includes high-level neural network layers, optimizers, distributed training primitives, quantization support, and model I/O (safetensors, GGUF). It is designed for efficient on-device ML research and inference, with an emphasis on performance via custom GPU kernels, JIT compilation, and unified memory.
llmfit is a comprehensive LLM model analysis and recommendation system that matches models to local hardware specifications. It detects system capabilities (CPU, RAM, GPU/VRAM), analyzes model requirements across multiple dimensions (memory fit, speed, quality, context), and provides recommendations for optimal model selection. The system features multiple interfaces including a terminal user interface (TUI) as the default mode, command-line interface, embedded web dashboard, and REST API. It supports various inference backends (Ollama, llama.cpp, MLX, Docker Model Runner, etc.), includes benchmarking capabilities to measure real-world performance, and enables users to share benchmark results with the community. The architecture follows a modular Rust workspace design with llmfit-core as the shared library containing hardware detection, model analysis, and provider abstractions, while interface-specific concerns are separated into llmfit-tui, llmfit-desktop, llmfit-web, and llmfit-python packages.
Einops is a Python library that provides a uniform, readable, and reliable API for tensor manipulation across multiple deep learning frameworks (NumPy, PyTorch, TensorFlow, JAX, and others). It uses Einstein-like notation (e.g., 'b c h w -> b h w c') to express axis reordering, composition, decomposition, reduction, and repetition in a single, explicit string. The library is built around a core transformation engine that parses axis patterns into a cached 'TransformRecipe', which is then applied via a backend abstraction layer. This architecture allows einops to support 10+ frameworks with minimal code duplication, while also providing framework-specific layer wrappers (Rearrange, Reduce, EinMix) for seamless integration into model definitions. The library is designed for production and research use, with a focus on reducing errors from manual view/permute/transpose operations.
Voicebox is a local-first, open-source desktop application for AI voice cloning, text-to-speech (TTS), speech-to-text (STT), and dictation. It provides a full voice I/O stack that runs entirely on the user's machine, supporting multiple TTS engines (Qwen3-TTS, Chatterbox, Kokoro, TADA, LuxTTS, etc.) with zero-shot voice cloning from short audio samples. The application is built as a Tauri v2 desktop shell (Rust) wrapping a React/TypeScript frontend and a Python FastAPI backend with SQLite storage. Key features include global dictation with auto-paste, voice profile management with effects chains, a multi-track story editor, MCP server integration for AI agents, and GPU acceleration (CUDA, ROCm, MLX, DirectML). The architecture follows a service-oriented pattern with a backend factory for pluggable TTS/STT engines, SSE-based progress streaming, and a platform abstraction layer for cross-platform compatibility. The codebase is well-structured with extensive documentation, CI/CD pipelines, and a large community.
Atomic Chat is a full-featured desktop application for running local large language models and connecting to cloud AI providers. Built with Tauri v2 (Rust backend) and a React/TypeScript frontend, it provides a unified chat interface, agent mode with tool execution, MCP server integration, model hub, and an OpenAI-compatible local API server. The application supports multiple inference backends (llama.cpp, MLX, Apple Foundation Models) and handles model downloads, hardware detection, and agent orchestration. It includes extensive testing, telemetry (PostHog/Sentry), and a plugin architecture for extensibility. The codebase demonstrates clean separation of concerns through a service hub pattern, consistent state management with versioned persistence, and robust resource handling for MCP lifecycle and grammar-safe tool calling. Some duplication exists between two llama.cpp provider plugins, but overall architecture is well-structured.
TimesFM is a pretrained decoder-only transformer foundation model for time-series forecasting, developed by Google Research. The repository provides multiple model versions (v1.0, v2.0, v2.5, v3.0) with implementations in PyTorch, Flax/JAX, and MLX (Apple Silicon). It supports zero-shot univariate and multivariate forecasting with covariates, quantile outputs, and various preprocessing options (RevIN normalization, linear detrending, symmetric averaging). The codebase includes a high-level forecaster API, CLI scripts for CSV forecasting and system checks, fine-tuning examples (LoRA/DoRA), and comprehensive benchmarks (GIFT-Eval, TIME, FEV). While primarily an application library, it also contains research-oriented components like benchmark runners and notebooks, making it a hybrid project with a strong application focus.
WhisperPlus is a Python library that provides enhanced speech processing capabilities built around Whisper models. It offers a modular pipeline architecture for speech-to-text transcription with optimizations like quantization and flash attention, text summarization (including long-text handling), speaker diarization, text-to-speech conversion, video auto-captioning, and retrieval-augmented generation for chatting with video content. The library integrates with Hugging Face Transformers, Apple MLX for optimized inference on Apple silicon, and various external services like YouTube downloading and vector databases for RAG functionality. The architecture follows a modular design with clearly separated pipelines for each functionality: core speech-to-text (whisper.py), summarization (summarization.py, long_text_summarization.py), text-to-speech (text2speech.py), diarization (whisper_diarize.py), auto-captioning (whisper_autocaption.py), and RAG chatbots (lancedb_chatbot.py, autollm_chatbot.py). Utility modules handle YouTube media downloading (download_utils.py) and text formatting (text_utils.py). The library provides both programmatic API access and a Gradio-based web interface (app.py) for interactive use, along with a CLI entry point (cli.py). Data flows typically begin with audio/video input (from files or YouTube URLs), proceed through specialized processing pipelines (transcription, summarization, etc.), and produce text, audio, or video outputs. For RAG features, transcripts are processed into vector stores enabling question-answering interactions. The library relies on established dependencies including torch, transformers, ffmpeg, pyannote.audio for diarization, and MLX-specific libraries for Apple silicon optimization, demonstrating practical integration of state-of-the-art models into an accessible toolkit.
PMetal is a comprehensive machine learning framework specifically optimized for Apple Silicon hardware. It provides a complete suite for LLM training, fine-tuning, inference, and deployment through multiple interfaces (CLI, TUI, GUI, Python SDK, and REST API). The framework leverages Apple-specific technologies like Metal GPU, ANE, and UltraFusion for performance optimization while supporting standard ML workflows including LoRA/QLoRA fine-tuning, knowledge distillation, model merging, and quantization to formats like GGUF. The architecture follows a modular workspace design with over 20 specialized crates handling different aspects of the ML lifecycle, unified through a job specification system and configuration framework.
exo is a distributed inference system for running large language models (LLMs) and image generation models across clusters of devices, primarily targeting Apple Silicon hardware via the MLX framework. It enables running models larger than a single device's memory by sharding them across multiple nodes using pipeline and tensor parallelism. The system provides an OpenAI-compatible API, a web dashboard, and a native macOS application. The architecture is event-sourced, with a master node managing state and workers executing inference tasks. Communication is handled by a custom Rust networking layer built on Zenoh, supporting peer discovery and RDMA over Thunderbolt. The codebase demonstrates strong engineering in several areas, including immutable state models (FrozenModel), typed pub/sub routing, and sophisticated KV-cache management for LLM inference. While the core architecture is sound, the project carries significant technical debt through custom forks of critical dependencies (zenoh, mlx-lm), extensive monkey-patching to work around upstream library limitations, and a highly complex Nix-based build system. The codebase is tightly coupled to the Apple Silicon ecosystem, limiting portability but enabling deep optimization. It is a high-quality, ambitious project under active development, but its complexity and ecosystem coupling present challenges for long-term maintainability and external contribution.
This repository is a collection of standalone examples demonstrating the MLX framework (Apple's machine learning framework for Apple Silicon). It covers a wide range of deep learning models including LLMs (Mistral, Mixtral, Llama, T5), image generation (Stable Diffusion, FLUX, CLIP), video generation (Wan2.1), audio (Whisper, MusicGen, EnCodec), segmentation (SAM), and more. Each example is self-contained with its own entry point (CLI or Python script), model conversion utilities, and often a training/fine-tuning script. The repository is primarily an application-focused showcase with some research elements (e.g., normalizing flows, GCN). Production entry points include `whisper/mlx_whisper/cli.py` for transcription, `flux/txt2image.py` for image generation, `video/wan2.1/txt2video.py` for video generation, and `lora/lora.py` for fine-tuning. All examples depend on the MLX library and often bundle conversion from PyTorch weights. The architecture is a flat collection of independent projects, not a unified framework, leading to some code duplication (e.g., T5 and EnCodec are copied into multiple directories). However, several components are well-crafted and reusable, notably the Whisper decoding pipeline, the LoRA adapter module, and the flow-matching samplers for diffusion models.