$ show onyx-dot-app/onyx-foss
Onyx is a full-stack AI enterprise search and chat platform. The backend is a Python/FastAPI service with Celery workers, PostgreSQL, Redis, and a vector search engine (Vespa/OpenSearch). It ingests documents from 50+ external connectors (Google Drive, Slack, Confluence, Jira, SharePoint, etc.), runs a chunking/embedding/indexing pipeline, and serves RAG chat and search to multiple frontends: Next.js web app, React Native/Expo mobile app, Tauri desktop wrapper, Go CLI, an embeddable web widget, and a Terraform provider. It also includes an MCP server, deep research flows, sandboxed code-execution sessions (Craft), prompt caching, SSO/OAuth/SCIM multi-tenancy, rate limiting, and an admin analytics console. The repository has a hybrid character because it contains substantial data-engineering machinery (indexing pipelines, chunkers, embedding providers, evals) but the dominant identity is a production application rather than a research project.
The LLM identified 5 files worth highlighting for reuse:
Implements paginated Google Drive/Gmail retrieval with a tenacity-based retry decorator that specifically handles transient connection failures (BrokenPipe, ConnectionReset, SSL EOF, timeout) and re-raises after exhaustion. The retry strategy is generic enough to be extracted for any unreliable external API and the pagination logic is isolated and testable.
Provides a reusable rate-limit wrapper for external API calls across connectors. This solves a recurring problem in data ingestionβavoiding 429s and respecting third-party API quotasβwithout coupling to any specific connector implementation.
Self-contained PKCE code verifier and challenge generator for OAuth flows. It has no framework-specific dependencies and can be dropped into any OAuth client or authorization server implementation needing RFC 7636 support.
Analyzes spreadsheet-like sections to drive intelligent chunking decisions, preserving row/column context when splitting tabular content for RAG ingestion. This addresses a common document-preprocessing problem and the analysis logic is adaptable to different tabular data sources beyond Onyx's connectors.
A tenacity-based retry wrapper around vector document index operations that adds exponential backoff and transient-failure tolerance without blocking callers. It demonstrates a clean defensive-integration pattern for external search engines and can be reused for any flaky remote index client.