$ show AlexsJones/llmfit
llmfit is a comprehensive LLM model analysis and recommendation system that matches models to local hardware specifications. It detects system capabilities (CPU, RAM, GPU/VRAM), analyzes model requirements across multiple dimensions (memory fit, speed, quality, context), and provides recommendations for optimal model selection. The system features multiple interfaces including a terminal user interface (TUI) as the default mode, command-line interface, embedded web dashboard, and REST API. It supports various inference backends (Ollama, llama.cpp, MLX, Docker Model Runner, etc.), includes benchmarking capabilities to measure real-world performance, and enables users to share benchmark results with the community. The architecture follows a modular Rust workspace design with llmfit-core as the shared library containing hardware detection, model analysis, and provider abstractions, while interface-specific concerns are separated into llmfit-tui, llmfit-desktop, llmfit-web, and llmfit-python packages.
The LLM identified 5 files worth highlighting for reuse:
Cross-platform hardware detection system that identifies CPU, RAM, and GPU specifications across Linux, Windows, and macOS. Handles unified memory architectures (Apple Silicon, AMD APUs, NVIDIA Grace) with multiple fallback paths (vendor tools → sysfs → estimation). Includes override capabilities for testing and extensive test suite. Demonstrates skill in robust system profiling with graceful degradation patterns.
Uniform abstraction layer over diverse LLM runtime providers (Ollama, MLX, llama.cpp, Docker Model Runner, LM Studio, vLLM, RamaLama). Defines ModelProvider trait with standardized interface for availability checking, installed model detection, and download/pull operations. Features unified PullHandle/PullEvent progress reporting system, name resolution helpers, and installation detection logic. Demonstrates skill in creating extensible abstractions over disparate APIs with consistent error handling and progress tracking.
Hardware-aware model fitting and scoring system that analyzes LLM compatibility with detected system specifications. Estimates token-per-second throughput based on hardware bandwidth, determines optimal execution paths (GPU, TensorParallel, MoeOffload, CPU offload, CPU-only), and provides multi-dimensional scoring (quality, speed, fit, context). Includes specialized handling for MoE models, pre-quantized models, and TTS models. Demonstrates skill in performance modeling with configurable parameters and domain-specific optimizations.
Benchmark sharing system that automates contribution of performance data to the community via GitHub pull requests. Handles authentication through GitHub device flow, creates properly formatted benchmark JSON files in the community directory, and implements idempotent file operations with dry-run support. Demonstrates skill in GitHub API integration, automated PR creation, and robust file system operations with proper error handling.
Centralized theming system for the terminal user interface that defines color palettes and styling rules. Supports multiple themes (default, dracula, solarized, etc.) with synchronized CSS properties for the web dashboard. Demonstrates skill in creating maintainable UI styling systems that work across different presentation layers (TUI and web) with consistent visual identity.