DevDocs is a web application that aggregates and serves API documentation for hundreds of programming languages, frameworks, and tools in a unified, searchable, offline-capable interface. The backend is a Ruby/Sinatra server that provides the static assets and a minimal API, while the frontend is a custom JavaScript single-page application with client-side search, keyboard navigation, and a service worker for offline access. The core data generation pipeline consists of Ruby scrapers that fetch documentation pages from official sources, apply a chain of filters (HTML cleaning, entry extraction, URL normalization, image optimization), and produce normalized JSON indexes and cached pages stored on disk. The architecture is largely monolithic with a strong separation between the scraping pipeline (lib/docs) and the frontend (assets/javascripts). While primarily an application, it contains a substantial data engineering component in its scraper/filter system, which is reusable and generalizable for other documentation aggregation projects.
RDocumentation 2.0 is the production Next.js frontend for rdocumentation.org, a searchable documentation browser covering every version of every R package published on CRAN and Bioconductor. It is a pages-router Next.js application (Next 14, React 18) deployed as a Docker container on ECS/Kubernetes with CircleCI. The application renders package pages with READMEs, function/topic documentation with syntax-highlighted examples and MathJax-rendered math, download statistics, GitHub repository metrics, collaborator profiles, and search results with pagination. It also implements numerous legacy redirect routes (e.g. campus/help, link/[topic], bare package/topic URLs) that resolve to canonical versioned URLs server-side, plus a local health-check endpoint used by container orchestration. The architecture is a server-rendered UI that delegates all data access to an external backend API at api.rdocumentation.org, supplemented by the GitHub GraphQL API (requiring a GITHUB_TOKEN) for repo star/fork/issue counts and cranlogs.r-pkg.org for monthly download data. Search and autocomplete are client-side, hitting the backend's Elasticsearch-backed search_functions and search_packages endpoints directly. The most interesting engineering resides in the lib/ layer: a custom R-version comparator (R versions are not semver; segments like "1.0.10" vs "1.2-3" need numeric segment comparison), a search query parser that recognizes both stats::lm and lm(stats) scoped-search formats, and client-side deduplication logic that collapses duplicate package/function versions while preserving relevance order — compensating for known quirks in the backend API (identical relevance scores tie-broken by insertion order, latest=1 flag dropping packages with null latest_version_id, function searches lacking topic documents for newest versions). Operational reality is documented honestly in docs/runbook.md: the UI is the most modern component in a larger legacy system. The backend API, worker, and Elasticsearch components date from 2015-2016 and run Node 8; the Depsy collaborator data source has been dead since 2018; and the frontend has minimal error handling when the API is unavailable. The UI code itself is well-structured and cleanly separated between components, lib helpers, and pages, but it ships without unit tests (only Playwright health checks), TypeScript is configured non-strict, and several dependencies lag their ecosystems (ESLint 7, TypeScript 4.9, react-markdown 5's deprecated plugin API, an old recharts beta).