RustyRAG is a production-grade RAG (Retrieval-Augmented Generation) API server written in Rust, designed for sub-200ms time-to-first-token. It provides a complete pipeline: document ingestion (PDF, DOCX, PPTX, XLSX, HTML, TXT, ZIP) via Docling with layout-aware extraction, OCR, table recognition, and optional vision-model image descriptions; hybrid search using Milvus (dense HNSW vectors + BM25 sparse with Reciprocal Rank Fusion); cross-encoder reranking via Jina Reranker v3; and streaming LLM responses from Groq or Cerebras. The server exposes REST endpoints for document upload, semantic search, RAG chat (both full response and SSE streaming), collection backup, and an evaluation benchmark runner. The architecture follows a clean layered pattern: Actix-web handlers in `src/handlers/` orchestrate business logic, services in `src/services/` encapsulate external API clients (Milvus, embeddings, reranker, Docling, LLM), and schemas in `src/schemas/` define request/response types with OpenAPI annotations via utoipa. The Rust binary is a single self-contained server; GPU-accelerated sidecars (embeddings, reranker, Docling) run as Docker containers. Configuration is environment-driven with sensible defaults. Key entry points are the HTTP API at `/api/v1/` (Swagger UI at `/swagger-ui/`) and a built-in chat UI at `/static/chat.html`. The primary data flow for a RAG query is: embed user query → fire concurrent dense and BM25 searches in Milvus → merge via RRF → rerank top candidates → build context prompt → stream LLM response with source citations and timing metadata. The system is designed for low latency and high accuracy, with benchmark results showing 94.5% pass rate with reranker at ~279ms TTFT.
NextChat is a production-grade, multi-provider AI chat client built as a Next.js web application with an optional Tauri desktop wrapper. It supports 15+ LLM providers (OpenAI, Anthropic, Google, Baidu, DeepSeek, etc.) through a unified API abstraction layer, and provides a rich chat interface with streaming responses, markdown rendering, code folding, image upload, TTS, real-time voice chat, and Stable Diffusion image generation. The application is designed for self-hosting (Docker, Vercel, manual) and emphasizes privacy by storing all chat data locally in IndexedDB/localStorage, with optional cloud sync via WebDAV or UpStash Redis. Architecturally, NextChat is a monolith with client-side state management using Zustand (with IndexedDB persistence), serverless API routes for proxying LLM requests, and a React component library. It includes a plugin system for OpenAPI-based function calling, MCP (Model Context Protocol) integration for external tool execution, and a prompt/mask system for reusable system instructions. The codebase is well-structured with reusable patterns like generic store factories, UI components, and streaming utilities, though it exhibits significant duplication across provider integrations and some features (e.g., Tencent, SD) appear less maintained.
Witsy is a feature-rich desktop AI assistant built with Electron and Vue 3. It operates as a Bring-Your-Own-Keys (BYOK) client, aggregating dozens of LLM providers (OpenAI, Anthropic, Google, Ollama, etc.) into a unified chat, agent, and automation platform. The architecture follows a standard Electron main/renderer process split, communicating via IPC, with a Vue 3 frontend and a Node.js backend. Beyond basic chat, Witsy provides a sophisticated agent system supporting multi-step workflows, A2A communication, scheduling, and webhooks. It includes a full RAG pipeline (document ingestion, chunking, embedding, vector search), a plugin system for tools (MCP, code execution, web browsing, image generation), and extensive voice capabilities (STT/TTS with multiple providers). Data is persisted locally as JSON files within workspaces, with a vector database for semantic memory and document retrieval. A key differentiator is its automation layer, which can control the desktop (text selection, pasting, app detection) via platform-specific automators (macOS, Windows, nut-js). It exposes a local HTTP API and a CLI for external integration. The MCP (Model Context Protocol) client allows it to connect to a vast ecosystem of third-party tools. While primarily an application, it contains significant data engineering components (RAG pipeline, memory manager, deep research orchestrator) that are tightly integrated into the app's workflow. The codebase is well-structured with a strong emphasis on testing, though its massive feature surface and reliance on external APIs introduce inherent complexity and maintenance challenges.