llmfit is a comprehensive LLM model analysis and recommendation system that matches models to local hardware specifications. It detects system capabilities (CPU, RAM, GPU/VRAM), analyzes model requirements across multiple dimensions (memory fit, speed, quality, context), and provides recommendations for optimal model selection. The system features multiple interfaces including a terminal user interface (TUI) as the default mode, command-line interface, embedded web dashboard, and REST API. It supports various inference backends (Ollama, llama.cpp, MLX, Docker Model Runner, etc.), includes benchmarking capabilities to measure real-world performance, and enables users to share benchmark results with the community. The architecture follows a modular Rust workspace design with llmfit-core as the shared library containing hardware detection, model analysis, and provider abstractions, while interface-specific concerns are separated into llmfit-tui, llmfit-desktop, llmfit-web, and llmfit-python packages.
Jan is an open-source desktop application that provides a ChatGPT-like interface for interacting with both local and remote AI models. Built with Tauri (Rust backend) and React (TypeScript frontend), it allows users to download, manage, and run LLMs locally via backends like llama.cpp and MLX, while also supporting cloud APIs from OpenAI, Anthropic, and others. The application features an OpenAI-compatible API server, custom assistants, agent capabilities ('Cowork' mode), Retrieval-Augmented Generation (RAG), web search, and Model Context Protocol (MCP) integration for extensibility. It emphasizes privacy and local processing while offering cross-platform support for Windows, macOS, Linux, iOS, and Android.
AnythingLLM is a full-stack, self-hosted AI chat platform designed for Retrieval-Augmented Generation (RAG) over private documents. It provides a web interface where users create workspaces, upload documents (PDF, DOCX, images with OCR, audio, etc.), which are then chunked, embedded, and stored in a vector database (default LanceDB, with support for PGVector, Pinecone, Qdrant, and others). Chat queries retrieve relevant document chunks and feed them to an LLM of choice (supporting 30+ providers including OpenAI, Anthropic, Ollama, and local models via KoboldCPP). The platform also features an extensible agent system (aibitat) with plugins for file system operations, SQL databases, Gmail, Google Calendar, web scraping, image generation, and scheduled jobs. A separate 'collector' microservice handles document ingestion and format conversion, while the 'open-computer' subproject provides a QEMU-based virtual machine for autonomous browser automation. The frontend is a React SPA with multi-user support, role-based access, internationalization, embeddable chat widgets, and a community hub for sharing agent skills, system prompts, and slash commands. The architecture is a monolith (Express server) with the collector as an isolated service, and the open-computer as a standalone CLI/service. It relies on external AI APIs, vector databases, and local storage, making it suitable for both cloud and on-premises deployment.