Paperless-ngx is a production-grade, self-hosted document management system. It ingests physical and digital documents through a consume folder, REST API uploads, and IMAP email fetching, then runs OCR via Tesseract/ocrmypdf, extracts text through a pluggable parser registry, generates archive PDFs and thumbnails, and stores structured metadata (tags, correspondents, document types, custom fields). Documents are indexed in Tantivy and exposed through a Django REST API consumed by an Angular SPA that provides search, editing, bulk operations, permissions, workflows, sharing, and an admin interface. Background processing is handled by Celery/Redis with real-time status pushed over WebSockets. The system also includes optional AI/LLM classification, RAG chat over the document index, and a workflow engine that reacts to consumption and document lifecycle events with actions like assignment, emailing, webhooks, and trash moves. Though primarily an application, it contains a substantial search-engine subsystem and an optional LLM/embedding pipeline, giving it a mild hybrid character.
Stirling PDF is a comprehensive, open-source PDF processing platform that provides 50+ document manipulation tools (merge, split, compress, OCR, sign, redact, convert, etc.) via a web UI, REST API, and desktop client (Tauri). The backend is a modular Java Spring Boot application split into core, common, proprietary, and SaaS modules, while the frontend is a React/TypeScript app with Mantine UI and Storybook. The system also includes a Python FastAPI engine for AI-powered features such as document classification, math auditing, and question answering, and an enterprise policy engine for automated document workflows with triggers, pipelines, and output sinks (local, S3, vector DB). It supports self-hosted, SaaS (Supabase/Stripe), and desktop deployment modes with clustering via Valkey/Redis backplane, SSO/OAuth2/SAML2 authentication, and comprehensive billing/licensing (PAYG, subscriptions, enterprise). The architecture is multi-module with clear separation of concerns: `app/common` provides shared utilities (PDF parsing, security, cluster abstractions, temp file management), `app/core` implements core PDF operations, `app/proprietary` adds enterprise features (encrypted storage, policy engine, audit, hardware signing), and `app/saas` handles SaaS-specific billing and team management. The frontend uses a shadowing pattern to swap implementations per build variant (core, proprietary, desktop, SaaS) without conditional imports. The AI engine runs as a separate Python service with streaming progress events and two-round protocols for complex tasks. Key data flows include: PDF upload → tool execution (Java PDFBox/iText) → output download; policy-triggered processing (folder watch, schedule) → pipeline execution → output delivery; AI agent workflows (classification, auditing) with progress streaming. External dependencies include LibreOffice (document conversion), Tesseract (OCR), PDFBox, iText, and various cloud services (Supabase, Stripe, PostHog). The codebase demonstrates high craftsmanship with extensive unit/integration/E2E tests, reusable component patterns (createToolFlow, useBaseTool), and well-abstracted interfaces for storage, signing, licensing, and clustering.
WeasyPrint is a production-grade Python library and CLI that renders HTML/CSS documents to PDF with high fidelity to print-focused web standards. In production it parses HTML via tinyhtml5, parses and validates CSS via tinycss2/cssselect2, builds a CSS box tree, runs full CSS layout algorithms for block, inline, table, flex, grid, multicolumn, floats, and paged media, then paints the result into a PDF via pydyf. It also renders embedded and standalone SVG, embeds and subsets fonts through Pango, HarfBuzz, Fontconfig, and fontTools, and supports PDF/A, PDF/UA, PDF/X, PDF forms, attachments, bookmarks, metadata, and e-invoice flavors such as Factur-X/ZUGFeRD. The main entry points are the `weasyprint` command-line tool (`weasyprint/__main__.py`) and the public Python API exposing `HTML`, `CSS`, `Document`, and `Page` (notably `HTML(...).write_pdf(...)`). Architecturally the repository is a layered pipeline rather than a monolith or microservice: CSS property validation and computed-value resolution feed a formatting-structure builder, which produces boxes; those boxes go through layout modules that handle pagination and page-breaking; a drawing layer converts the laid-out box tree into PDF stream operations; and the PDF package assembles final file objects, metadata, and conformance variants. The code is heavily specification-driven, with many docstring references to CSS 2.1, CSS Paged Media, CSS Flexbox/Grid, SVG, and PDF standards. Extensive pixel-comparison tests, regression tests tied to GitHub issues, and CI workflows for PDF/A and PDF/UA validation back the implementation. External runtime dependencies include pydyf, tinyhtml5, tinycss2, cssselect2, Pyphen, Pillow, fontTools, and native Pango/HarfBuzz/Fontconfig libraries; Ghostscript is used only by the test suite for rendering comparisons. The codebase is mature and generally well-structured, with clean separation between CSS processing, layout, drawing, and PDF output. It contains some inevitable complexity around pagination, table layout, and font handling, plus a set of `xfail` tests and TODO comments marking known spec coverage gaps, but overall it is one of the strongest open-source HTML-to-PDF engines and is actively maintained with recent security fixes and broad standards support.
Asciidoctor Web PDF is a CLI tool that converts AsciiDoc documents to PDF using web technologies. It employs a multi-stage pipeline: Asciidoctor converts .adoc to HTML, Puppeteer renders the HTML via the Vivliostyle Viewer (CSS paged media), and pdf-lib adds PDF outline/bookmarks and metadata. The architecture features clean separation of concerns with dedicated modules for browser management, MathJax processing, syntax highlighting, outline generation, and metadata injection. Notable for its sophisticated handling of Node.js SEA binary execution paths and server-side MathJax rendering with proper async font loading.
This repository is a production-grade, canvas-based WYSIWYG rich text document editor published as the npm package `@hufe921/canvas-editor`. It is a monolithic library that provides a fully self-contained document editing experience, rendering text, tables, images, LaTeX, and form controls entirely on an HTML `<canvas>` element. The architecture is a well-structured layered design featuring a Command pattern facade (`Command`/`CommandAdapt`), a custom particle-based rendering engine (`Draw`), a pub/sub event system (`EventBus`), and a plugin system. The editor supports a wide array of features including pagination, headers/footers, watermarks, change tracking (trace mode), macros, spell checking, accessibility, multi-column layouts, and a sophisticated cascade expression engine for dynamic form control behavior. It is specifically designed for high-fidelity document editing in domains like Electronic Medical Records (EMR), contracts, and official documents, as evidenced by the mock data and feature set. The codebase demonstrates strong engineering craftsmanship. Several components are genuinely reusable, including a custom Myers O(ND) diff algorithm, a standalone LaTeX-to-SVG renderer using Hershey fonts, a fully featured DSL expression evaluator for control cascading, and a clean event bus implementation. The project is actively maintained with a comprehensive test suite (Cypress E2E, Vitest unit tests), modern tooling (Vite, TypeScript), and a rich plugin ecosystem. While some core files like `Draw.ts` and `CommandAdapt.ts` are large and exhibit tight coupling, the overall architecture, separation of concerns, and quality of the reusable components are well above average.
MarkItDown is a Python library and CLI tool that converts a wide variety of file formats (PDF, DOCX, PPTX, XLSX, HTML, CSV, JSON, EPUB, Outlook MSG, images, audio, YouTube, RSS, IPYNB, ZIP, and more) into Markdown. Its primary production use is preparing documents for LLM ingestion, agent workflows, and text analysis pipelines. The architecture is a monorepo containing the core `markitdown` package, an OCR extension (`markitdown-ocr`) that uses LLM Vision for image text extraction, an MCP server (`markitdown-mcp`) exposing conversion as a tool for AI agents, and a sample plugin for third-party converters. The core library uses a plugin system based on `importlib.metadata.entry_points` to register converters, each implementing a `DocumentConverter` base class with `accepts()`/`convert()` methods and a priority-based dispatch. The main entry points are the `MarkItDown` class (Python API), the `markitdown` CLI, and the MCP server's `convert_to_markdown` tool. Data flows from local files, streams, HTTP responses, or URIs through converter selection (by MIME type, extension, or content inspection) to produce Markdown output. External dependencies include many optional format-specific libraries (pdfminer.six, python-docx, python-pptx, openpyxl, etc.), external binaries (exiftool, ffmpeg), cloud APIs (Azure Document Intelligence, Azure Content Understanding), and an OpenAI-compatible LLM client for image captioning and OCR.
Docling is a production-grade Python library for converting diverse document formats (PDF, DOCX, PPTX, XLSX, HTML, LaTeX, EPUB, images, audio, video, email, EBCDIC, and more) into a unified, structured `DoclingDocument` representation. It is designed to serve as the document understanding layer for AI workflows, particularly Retrieval-Augmented Generation (RAG), agentic AI, and document extraction pipelines. The system provides a rich Python SDK, a Typer-based CLI (`docling convert`), a service client for remote conversion via `docling-serve`, and an MCP server. It supports multiple output formats including Markdown, JSON, HTML, DocTags, and plain text. Architecturally, Docling employs a modular pipeline system with pluggable backends for each input format and a stage-based processing model. The core pipeline (`StandardPdfPipeline`, `VlmPipeline`, `AsrPipeline`) orchestrates page-level operations (preprocessing, OCR, layout detection, table structure recognition, picture classification, reading order) and document-level enrichments (code/formula extraction, heading hierarchy, picture description). Inference engines are abstracted behind factory patterns supporting ONNX Runtime, HuggingFace Transformers, vLLM, MLX, KServe v2, and remote APIs. The library integrates with over 20 OCR engines and vision-language models, and includes a plugin system for extensibility. Key entry points include the `DocumentConverter` class, the CLI (`docling.cli.main`), and the service client (`docling.service_client`). External dependencies span PDF libraries (pypdfium2, docling-parse), ML frameworks (torch, transformers, onnxruntime), OCR engines (Tesseract, EasyOCR, RapidOCR), and cloud services (KServe, Azure, S3). The codebase is actively maintained with frequent releases, extensive test coverage, and a clear migration path from legacy components.
PDFMathTranslate is a production-grade CLI and GUI application for translating PDF documents while preserving scientific formatting (formulas, charts, tables, and fonts). It uses an ONNX-based layout detection model (DocLayout-YOLO) to understand document structure, supports OCR for scanned PDFs, and integrates with numerous translation services (Google, DeepL, OpenAI, Ollama, etc.). The system operates as a monolithic package with multiple entry points: a command-line tool, a Gradio web interface, a Flask+ Celery HTTP API, and an MCP server for AI assistant integration. Internally, it follows a pipeline architecture: input PDF → optional OCR → layout analysis → text extraction → translation (with caching) → typesetting → bilingual output PDF. Notable engineering patterns include a pluggable translation backend via a kernel registry, a configurable translator base class with retry and caching, and a singleton configuration manager. The code is well-structured with test coverage and CI/CD pipelines, though the dependency on a Git submodule for the 'precise' kernel introduces minor coupling risk.
OCRmyPDF is a mature, production-grade command-line tool and Python library that adds searchable OCR text layers to scanned PDFs, producing PDF/A-compliant output by default. The system ingests PDFs or images, optionally deskews/cleans pages via unpaper, rasterizes with Ghostscript or pypdfium2, runs Tesseract OCR, grafts the recognized text into the original PDF using a plugin-based pipeline, and performs image optimization with pngquant/jbig2enc. It supports multiple output formats (PDF/A-1b, PDF/A-2b, plain PDF), a plugin system for replacing OCR engines, rasterizers, and progress bars, and includes a directory watcher for batch processing. Architecture is a modular Python CLI/library with external subprocess orchestration; core abstractions include typed OcrOptions (pydantic), Executor concurrency, FontProvider chains, and the OcrElement data model. The codebase is exceptionally well-tested with a comprehensive suite of unit, integration, and plugin-level tests, and demonstrates advanced patterns for PDF repair, cross-platform tool discovery, and atomic file operations.
OpenDataLoader-PDF is a production-grade, multi-language PDF-to-structured-content extraction tool designed for AI/LLM pipelines, RAG systems, and accessibility workflows. It converts PDFs into JSON, Markdown, HTML, plain text, and tagged PDF (PDF/UA) with high accuracy (0.907 overall benchmark). The core engine is written in Java (veraPDF-based) and provides a rich CLI, while Python and Node.js SDKs bundle the Java JAR and expose idiomatic APIs. A hybrid mode routes pages to an external AI backend (Docling) for OCR, formula extraction, and picture descriptions, with a triage processor deciding per-page routing based on content signals. The project also includes an MCP server for AI agent integration, a LangChain loader, and an agent skill bundle for correct usage.
Sioyek is a cross-platform PDF viewer optimized for reading research papers and technical books. Built with Qt, OpenGL, and MuPDF, it provides a rich set of features including multi-threaded rendering, annotation management (highlights, bookmarks, portals, freehand drawings), SyncTeX integration for LaTeX forward/inverse search, fuzzy text search, and a customizable command-driven interface. The architecture follows a modular design with a central MainWidget orchestrating UI state, a Document model for PDF handling, a PdfRenderer for threaded rendering, and a DatabaseManager for persistent annotation storage. The application supports both desktop and touch-optimized interfaces, with platform-specific code for Windows, macOS, Linux, and Android. While primarily an application, Sioyek incorporates data-oriented features such as SQLite-based annotation persistence, checksum-based document identification, and integration with external research tools (Google Scholar, arXiv). The repository also includes scripts for paper downloading, text-to-speech, and embedding annotations, as well as precomputed embedding data files (embedding.npy, linear.npy) likely used for text summarization or search enhancement. The codebase demonstrates strong engineering practices with reusable components like a fuzzy matching library (fzf), a SyncTeX parser, a type-safe configuration system, and a set of QML touch UI widgets. However, it also exhibits some technical debt in the form of global state, large monolithic files (main_widget.h), and platform-specific conditionals scattered throughout.
OneFileLLM is a versatile CLI tool and web application designed to process diverse inputs (local files, GitHub repositories, YouTube transcripts, arXiv papers, web pages, etc.) into a single structured XML-like output optimized for LLM consumption. The system features format-specific processors that extract and convert content (e.g., Excel to markdown, PDF text extraction, Jupyter notebook to code), token counting via tiktoken with optional NLTK-based compression, and metadata reporting. It includes a Rich-powered terminal interface with colored output and progress bars, a Flask-based web app with Server-Sent Events for real-time progress updates, and an alias system for JSON-stored shortcuts with placeholder substitution. Security is prioritized through SSRF protection, local path validation, and offline mode support. The architecture separates core processing logic (onefilellm.py), CLI handling (cli.py), utilities (utils.py), and web components (extras/web_app.py), enabling modular reuse.
This repository provides safe, idiomatic Rust bindings to the MuPDF C library (Artifex Software), enabling programmatic reading, rendering, creation, and editing of PDF and other document formats. It is a library crate organized as a Cargo workspace with three sub-crates: `mupdf` (high-level API), `mupdf-sys` (FFI layer that compiles MuPDF from source), and optional font crates (`mupdf-fonts-droid`, `mupdf-fonts-noto`, `mupdf-fonts-sil`). The architecture is a classic FFI wrapper: `mupdf-sys` exposes raw C functions via `bindgen`, while `mupdf` provides safe Rust types (`Document`, `Page`, `Pixmap`, `PdfDocument`, `Shape`, etc.) with automatic resource management, error handling, and thread-safety checks. Key entry points include `Document::open()` for loading files, `Page::to_pixmap()` for rasterization, `Shape` for programmatic drawing, and `PdfDocument` for PDF-specific operations like annotation manipulation, form filling, and incremental saving. The library supports extensive feature flags (e.g., `js`, `xps`, `svg`, `tesseract`, `docx-output`) that control which MuPDF subsystems are compiled. The codebase demonstrates high craftsmanship through its safe wrapper patterns, builder-style APIs (e.g., `Shape`), extensible trait design (`FontLoader`, `NativeDevice`), and comprehensive test coverage including snapshot-based rendering tests and link extraction round-trip tests. It is not a data research project; it is a software library for document processing.
ClawPDF is a zero-dependency PDFium WebAssembly library and CLI for Node.js and browsers. It extracts text, renders pages to RGBA bitmaps, and produces PNG bytes without runtime dependencies, native addons, or postinstall scripts. The library supports multiple input sources (file paths, URLs, Blobs, byte arrays) with configurable HTTP fetch timeouts and size budgets, password-protected PDFs, and text-first extraction with bounded image fallback. The CLI provides text extraction, JSON output, page rendering, and inline terminal image display for Kitty and iTerm2 protocols. The architecture is modular with clean separation of concerns: input normalization, engine lifecycle, document/page abstractions, render size resolution, PNG encoding, and extraction orchestration. The codebase demonstrates high craftsmanship with reusable components like the standalone PNG encoder and terminal inline image renderer.
PDFsam Basic is a mature, open-source JavaFX desktop application for splitting, merging, rotating, extracting, and mixing PDF files. It is built as a multi-module Maven project using JPMS modules, with the Sejda SDK as the PDF processing engine, EventStudio for event-driven communication, and a custom dependency injection framework. The application provides a rich GUI with tool panels, workspace persistence, preferences, theming, internationalization, update/news services, and premium tool integration. This is purely an application repository; there are no data research or analytics components.
MinerU is a production-grade document parsing engine that converts PDFs, images, and Office documents (DOCX, PPTX, XLSX) into structured Markdown and JSON. It supports multiple backends: a fast pipeline (layout analysis + OCR + formula/table recognition), a high-accuracy hybrid mode combining VLM and OCR, and a pure VLM mode using vLLM/LMDeploy. The system is deployed as a CLI tool, FastAPI service, Gradio web UI, or Docker container, with integrations for RAG frameworks like LangChain, LlamaIndex, and Dify. The codebase is well-architected with clean abstractions for data I/O (file/S3), model initialization (singleton pattern), and output generation (client-side rendering). It handles 109 languages and includes specialized modules for mathematical formula recognition (UniMERNet), table structure recovery (SLANet+/UNet), and office document normalization. While the code is actively maintained and functionally rich, it carries some technical debt from its PaddleOCR heritage (vendored code, eval-based builders) and has a complex optional dependency matrix for hardware accelerators.
This repository is the endroid/qr-code PHP library for generating QR codes in multiple output formats including PNG, GIF, WebP, SVG, EPS, PDF, binary text, and console output. It provides a clean object-oriented architecture with immutable value objects (QrCode, Color, Matrix), a Builder facade for convenient construction, and a set of Writer classes each implementing WriterInterface. The library delegates low-level QR encoding to BaconQrCode and handles matrix rendering, logo overlays, and label placement. It includes optional validation by decoding the generated image. The code is modern PHP 8.4, utilizing readonly classes, enums, named arguments, and strict typing. The separation of concerns is clear: QrCode holds data parameters, Matrix encapsulates the binary grid and size calculations, Writers produce format-specific results, and Results provide uniform access to the output. The Builder aggregates all common options and allows per-call overrides. The library is well-maintained with active CI, zero dependabot alerts, and runs on the latest PHP versions.
This is an Android PDF viewer application built with Jetpack Compose that uses Mozilla's pdf.js library for rendering. The app displays PDFs in a sandboxed WebView with strict security policies, handles encrypted PDFs, provides navigation controls, outline viewing, document properties, and various UI features like zoom, rotation, and text layer visibility. The architecture follows MVVM with a ViewModel managing state and a Compose UI layer. The app uses content providers to securely load PDF content without granting the WebView access to files or network.
pdfme is a TypeScript monorepo for template-driven PDF generation and manipulation. In production it does three things: (1) generates PDF bytes from a JSON template plus input data using a plugin-based schema renderer (packages/generator + packages/schemas), (2) ships React widgets — Designer, Form, Viewer — for visually editing templates, filling forms, and previewing output (packages/ui), and (3) exposes a CLI (packages/cli) for headless generation, validation, diagnostics, and PDF conversion, plus a page-manipulation library (packages/manipulator) and an experimental JSX authoring layer (packages/jsx). The low-level PDF engine is a heavily extended, vendored fork of pdf-lib (packages/pdf-lib) with custom AcroForm appearance providers, SVG drawing, and font subsetting; fontkit provides glyph metrics, bwip-js renders barcodes, and @napi-rs/canvas powers Node-side raster conversion. A playground web app and Docusaurus documentation site are included as consumers. The core data flow is: Template + inputs are validated by shared Zod schemas (packages/common), then each schema plugin's pdfRender implementation draws into a pdf-lib document while a dynamic-layout engine computes split ranges, overflow expansion, and table/list pagination across pages. The same Plugin contract (pdf/ui/propPanel/icon) drives both PDF rendering and the WYSIWYG interface, so custom field types register once and work everywhere. Notable engineering: a sandboxed expression evaluator with prototype-pollution and XSS guards (common/expression.ts), SSRF-safe URL fetching with IPv4/IPv6/mapped-address checks (common/helper.ts), a from-scratch UAX #14 line-breaking engine with Japanese kinsoku handling (schemas/text/lineBreak.ts), and a safe inline-Markdown-to-rich-text parser. Weaknesses: the vendored pdf-lib fork contains dormant/commented API stubs and diverges from upstream, image-snapshot tests are brittle across rendering environments, several modules use module-level mutable caches, and some package version pins are aspirational (e.g., TypeScript ^7 in website). Repo-wide version coordination relies on internal packages pinned at 0.0.0 with * workspace deps. Overall the codebase is professional, well-tested (unit, integration, e2e, visual snapshots, security regression tests), actively maintained with CI publishing, and is an application — not a data/research project.
This repository provides a fully-featured, cross-platform document reader and annotator for PDF, EPUB, and HTML snapshots, used as the core reading interface within the Zotero reference management ecosystem. It integrates PDF.js and EPUB.js for rendering, and a custom Structured Document Text (SDT) pipeline for cross-format search, reading mode (reflow), and read-aloud (TTS) with both browser-native and remote voice providers. The architecture is modular: a platform-agnostic common layer defines core abstractions (View, AnnotationManager, FocusManager, KeyboardManager, ReaderUI), with platform-specific bindings for Zotero desktop, web, iOS WebKit, and Android. The UI is built with React and features annotation tools (highlight, underline, note, ink, image, text), sidebar with thumbnails/annotations/outline, find-in-document via a web worker, and extensive keyboard navigation. Annotation management supports undo/redo, debounced saving, filtering, and export. The codebase demonstrates strong engineering practices with reusable design patterns (LRU cache, approximate string matching, navigation history, popup delay), but also contains large monolithic controller classes (Reader, PDFView) that handle many responsibilities. Overall, it is a high-quality, actively maintained application with sophisticated document processing capabilities.