$ show opendataloader-project/opendataloader-pdf
OpenDataLoader-PDF is a production-grade, multi-language PDF-to-structured-content extraction tool designed for AI/LLM pipelines, RAG systems, and accessibility workflows. It converts PDFs into JSON, Markdown, HTML, plain text, and tagged PDF (PDF/UA) with high accuracy (0.907 overall benchmark). The core engine is written in Java (veraPDF-based) and provides a rich CLI, while Python and Node.js SDKs bundle the Java JAR and expose idiomatic APIs. A hybrid mode routes pages to an external AI backend (Docling) for OCR, formula extraction, and picture descriptions, with a triage processor deciding per-page routing based on content signals. The project also includes an MCP server for AI agent integration, a LangChain loader, and an agent skill bundle for correct usage.
The LLM identified 7 files worth highlighting for reuse:
Rule-based PII replacement operating at TextLine/TextChunk granularity. Supports configurable sanitization rules (email, phone, IP, credit card) with overlap handling and bounding box updates. Reusable for any text sanitization pipeline requiring content safety.
Extensive set of regex patterns and heuristics for detecting bullet/list labels, including Arabic, Roman numerals, Korean, and Unicode bullets. Used by the list processor to identify list items. Standalone utility that can be extracted for any document list detection task.
Configurable content safety filter with default sanitization rules (email, phone, IP, credit card). Provides a clean builder/configuration pattern for enabling/disabling filters. Reusable as a template for any content filtering configuration in document processing.
Logs per-page hybrid routing decisions (JAVA/BACKEND) along with signal values and summary to a JSON file. Designed for benchmark evaluation and debugging of triage accuracy. Reusable pattern for recording and analyzing routing decisions in any multi-backend system.
Node.js script that generates CLI option definitions and convert functions for Python and Node.js from a single options.json source of truth. Demonstrates a robust codegen pattern to keep multi-language SDKs in sync. Reusable for any project needing synchronized option definitions across languages.
Per-element metadata sidecar carrying AI scores, source labels, TSR results, and caption info. Uses @JsonInclude(NON_DEFAULT) for clean JSON output. Reusable DTO pattern for attaching metadata to extracted document elements in any structured extraction pipeline.
Implementation of the XY-Cut++ reading order algorithm (arXiv:2504.10258) with cross-layout detection, density ratio, and adaptive axis selection. Handles multi-column layouts, headers/footers, and cross-layout elements. Reusable as a standalone reading order component for document layout analysis.