PageIndex is a production-ready RAG framework that replaces vector databases with a reasoning-based, hierarchical tree index over documents. Its core innovation is the 'flash' pipeline: a sophisticated, LLM-free PDF structure extractor that uses character-level layout analysis, trie-based keyword matching, and heuristic classification to build a table of contents tree from PDFs in seconds. The system then enables agentic retrieval by feeding this tree to an LLM, which reasons over the hierarchy to find relevant pages without embedding or chunking. The repository provides a full Python SDK (PageIndexClient) with local and cloud modes, CLI tool, integrations with OpenAI Agents SDK, Anthropic SDK, and Claude Agent SDK, plus an MCP bridge for remote tool execution. While primarily an application, it contains a deep research-grade data processing pipeline (flash) with extensive multilingual dictionaries and layout analysis heuristics, giving it a hybrid character. The codebase is well-structured with clear module boundaries, extensive test coverage, and reusable patterns like the tool contract system and trie-based matching. However, it carries medium technical drift risk due to hardcoded thresholds, dependency on private APIs of LLM frameworks, and pdfium version sensitivity.
Onyx is a comprehensive enterprise AI platform that provides knowledge retrieval, chat, and document indexing across 50+ third-party sources. The codebase reveals a FastAPI backend with Celery workers, PostgreSQL with Alembic migrations, OpenSearch vector search, Redis caching, and a separate model server. The frontend is a Next.js application with a custom design system (opal), alongside React Native mobile app, Go CLI, Chrome extension, and an embeddable widget. The platform supports deep research, custom agents, MCP servers, sandboxed code execution (Craft), and enterprise features like SSO, SCIM, billing, and multi-tenancy. Architecture is well-structured with clean separation of concerns, but the large codebase shows some drift: platform duplication across web/mobile/CLI, extensive Alembic migration history, and lint debt. The connector framework and design system are high-quality reusable components.