$ show zylon-ai/private-gpt
PrivateGPT is a production-grade, private AI assistant platform that provides RAG (Retrieval-Augmented Generation) over ingested documents, tool/function calling, sandboxed code execution, web search, MCP (Model Context Protocol) integration, skill management, and multimodal support. It is built as a FastAPI server with background workers (Celery and ARQ) for asynchronous ingestion and task processing, and exposes a Claude API-compatible streaming interface. The architecture is modular: a core chat engine with an interceptor pipeline for request/response processing, a pluggable component system with registries for LLMs, embeddings, readers, vector stores, and sandboxes, and a two-tier scheduling system (local and distributed) for tool and chat execution. The codebase demonstrates strong engineering practices with dependency injection, registry patterns, and composable interceptors, but its dual scheduling backends, extensive monkey-patching of third-party libraries, and heavy reliance on llama-index introduce maintenance complexity and technical drift. Reusable highlights include the context stack builder for dynamic system prompt assembly, the interceptor chain pattern, the sandbox abstraction for secure code execution, and the citation parser with incremental streaming support. The project is actively maintained by Zylon and powers their enterprise offering, with 57k+ stars and frequent releases.
The LLM identified 4 files worth highlighting for reuse:
Implements an immutable, layered context stack for assembling system prompts, tools, documents, and mounts per request. Uses discriminated-union layers with priority-based sorting and conflict detection. This pattern cleanly separates concerns from multiple sources (system prompts, tools, skills, documents) and can be extracted to any LLM application that needs dynamic, composable prompt construction without tight coupling to the hosting framework.
Defines a composable interceptor chain for chat request/response processing. Interceptors implement `ChatRequestLoopInterceptor` or `ChatResponseLoopInterceptor` interfaces and are executed sequentially by phase (initialization, preprocessing, LLM, postprocessing). The chain supports idempotent re-injection and is testable in isolation. This pattern is reusable in any request-processing pipeline that needs modular, cross-cutting concerns without framework dependency.
Provides an abstract `SandboxProvider` and `SandboxSession` for safe code execution. It decouples code execution from the environment via mount management and environment hydration, supporting local and remote backends. The abstraction includes session lifecycle, resource cleanup, and path translation. This is a well-crafted isolation pattern that can be reused in any system requiring sandboxed execution of untrusted code (bash, Python) with configurable mounts and environment variables.
Implements an incremental citation extractor that parses streaming text for source citations, handling edge cases like nested brackets, backtick-wrapped markers, and chunk boundaries. Uses a stateful parser that emits citations as soon as they are complete (for streaming events) and recovers from malformed input. This is a non-trivial NLP pattern that can be extracted to any system requiring real-time citation or reference extraction from streaming LLM output.