$ show opendatalab/MinerU
MinerU is a production-grade document parsing engine that converts PDFs, images, and Office documents (DOCX, PPTX, XLSX) into structured Markdown and JSON. It supports multiple backends: a fast pipeline (layout analysis + OCR + formula/table recognition), a high-accuracy hybrid mode combining VLM and OCR, and a pure VLM mode using vLLM/LMDeploy. The system is deployed as a CLI tool, FastAPI service, Gradio web UI, or Docker container, with integrations for RAG frameworks like LangChain, LlamaIndex, and Dify. The codebase is well-architected with clean abstractions for data I/O (file/S3), model initialization (singleton pattern), and output generation (client-side rendering). It handles 109 languages and includes specialized modules for mathematical formula recognition (UniMERNet), table structure recovery (SLANet+/UNet), and office document normalization. While the code is actively maintained and functionally rich, it carries some technical debt from its PaddleOCR heritage (vendored code, eval-based builders) and has a complex optional dependency matrix for hardware accelerators.
The LLM identified 6 files worth highlighting for reuse:
Provides a thread-safe context manager for PDFium document and page lifecycle. Uses threading.RLock to guard resource acquisition and release, with robust page-index probing (get_loadable_pdfium_page_indices) that handles corrupt PDFs gracefully. The pattern is self-contained and can be extracted for any project needing safe PDFium resource management.
Defines abstract base classes DataReader and DataWriter with a clean interface (read, write, list, is_dir, etc.). The filebase.py and s3.py implementations provide concrete file and S3 backends with lazy boto3 import. This abstraction decouples the parsing pipeline from storage, making it easy to add new backends (e.g., HTTP, GCS) without modifying core logic.
Implements normalize_docx_package, a robust function that repairs corrupted or non-standard DOCX (OOXML) packages before processing. It handles best-effort ZIP member reads, skips corrupt media, repairs relationship graphs, translates strict-to-transitional OOXML URIs, and rewrites the ZIP with CRC recomputation. The same pattern is replicated for PPTX and XLSX, demonstrating a reusable approach to OOXML package normalization.
Contains PPHGNetV2 backbone variants (B0-B6) with DiverseBranchBlock reparameterization blocks and the TheseusLayer base class. TheseusLayer provides pattern-based sub-layer replacement, output hooks, and freeze-before/stop-after utilities—a reusable pattern for building modular neural network backbones. The reparameterization logic (transI_fusebn, re_parameterize) is self-contained and can be extracted for any project needing efficient inference via structural reparameterization.
Implements UnimerMBartAttention with qk_squeeze (reduced key/query dimension) and modular attention variants (FlashAttention2, SDPA). The AttentionMaskConverter class generates causal + sliding window 4D masks with parallel step support. This pattern is reusable for any transformer-based model requiring efficient attention with custom masking, and the modular attention design allows easy swapping of implementations.
Contains GradioRequestConcurrencyLimiter, a generic async concurrency limiter with a wait queue and snapshot mechanism. It wraps Gradio's async event handlers to enforce a maximum number of concurrent requests, providing status updates to the UI. This pattern is reusable for any Gradio app needing rate limiting or queue management, and is implemented with clean async/await semantics.