OneFileLLM is a versatile CLI tool and web application designed to process diverse inputs (local files, GitHub repositories, YouTube transcripts, arXiv papers, web pages, etc.) into a single structured XML-like output optimized for LLM consumption. The system features format-specific processors that extract and convert content (e.g., Excel to markdown, PDF text extraction, Jupyter notebook to code), token counting via tiktoken with optional NLTK-based compression, and metadata reporting. It includes a Rich-powered terminal interface with colored output and progress bars, a Flask-based web app with Server-Sent Events for real-time progress updates, and an alias system for JSON-stored shortcuts with placeholder substitution. Security is prioritized through SSRF protection, local path validation, and offline mode support. The architecture separates core processing logic (onefilellm.py), CLI handling (cli.py), utilities (utils.py), and web components (extras/web_app.py), enabling modular reuse.
Local Deep Research is a self-hosted, privacy-focused AI research assistant that performs deep, agentic research using multiple LLMs and web searches. It provides a full-stack web application (FastAPI + JavaScript), a programmatic API, and a Model Context Protocol (MCP) server. The system ingests user queries, decomposes them, executes parallel searches across web and academic sources (SearXNG, Google, arXiv, PubMed, Semantic Scholar, etc.), fetches and extracts content, evaluates evidence, and synthesizes reports with citations. It features per-user encrypted SQLCipher databases, a settings system with environment variable overrides, a RAG library with FAISS vector store, journal quality scoring, news subscriptions, and extensive security controls (egress policy, SSRF protection, credential scrubbing). The codebase is heavily tested with unit, integration, UI, accessibility, and security tests. It is a hybrid of a web application and a research data pipeline, but the dominant character is a production-grade application.