BabelDOC is a production-grade PDF translation library and CLI tool that translates documents while preserving original layout, fonts, formulas, and styles. Its core architecture revolves around a custom Intermediate Representation (IL) that decouples PDF parsing from rendering, enabling a modular pipeline of parsing, layout analysis, LLM-based translation, typesetting, and PDF generation. The system supports complex PDF features including CJK text, Type3 fonts, tables, and formulas, and offers both monolingual and dual-language output modes. The translation pipeline is driven by an OpenAI-compatible LLM, with sophisticated prompt engineering for context-aware paragraph translation, glossary integration, and formula/style preservation. Layout analysis uses a DocLayout-YOLO ONNX model for detecting text blocks, tables, and figures. The codebase features a dual parser architecture, retaining a modified pdfminer for compatibility while developing a new native parser with improved font handling and security (CMap pickle deserialization hardening). BabelDOC demonstrates strong software engineering with several genuinely reusable components, including its typed IL data model, a priority thread pool executor, cross-platform memory monitoring utilities, and a high-performance glossary matching system using Hyperscan. The project is actively maintained with frequent releases, comprehensive CI/CD, and responsive security patching. While its complexity and dual-parser architecture present maintenance challenges, the overall code quality, architectural decisions, and reusable patterns make it a valuable reference for PDF processing and LLM-integrated document workflows.
TranslateBooksWithLLMs is a full-stack web application for translating eBooks (EPUB, DOCX, TXT, SRT) using multiple LLM providers (Ollama, OpenAI, Gemini, Mistral, DeepSeek, Poe, OpenRouter, NVIDIA NIM, LiteLLM). The architecture is a Flask server with Socket.IO for real-time progress streaming, a JavaScript SPA frontend with i18n support across 7 locales, and a SQLite-based checkpoint/resume system for interrupted translations. The core pipeline extracts content with HTML/XML placeholder preservation, chunks text using tiktoken-aware splitting, translates via configurable LLM providers with multi-phase fallback (LLM → token alignment → original), and reconstructs the output file. Additional features include glossary management with NER-based term suggestion, style extraction from source text, TTS generation via Edge-TTS and Chatterbox, and a comprehensive benchmark framework for evaluating translation quality across 35+ languages. The codebase demonstrates strong engineering with clean abstractions (adapter pattern for formats, factory pattern for LLM providers, generic orchestrator for translation pipeline), extensive testing (unit, integration, characterization with golden snapshots, e2e), and several genuinely reusable components. Notable patterns include a token-aware chunker with exact reassembly, an API key pool with round-robin rotation on 429 responses, an ordered concurrent executor that preserves chunk ordering, a retry manager with exponential backoff and circuit breaker, and a placeholder system for preserving HTML/XML structure during LLM translation. The benchmark subsystem and prompt optimization tool are supporting features that add research value but are secondary to the core application purpose.
This repository is a CLI tool for translating books (EPUB, TXT, PDF, SRT, Markdown) into bilingual versions (original + translation side-by-side) using large language models (OpenAI, Anthropic, Gemini, etc.) and free machine translation engines. It supports multiple translation backends, a provider configuration system, and a PDF-to-EPUB pipeline using docling for PDF extraction and Pandoc for EPUB export. The architecture is modular: loaders parse input formats, translators handle API calls with session context and capability probing, and a pipeline orchestrates PDF-to-EPUB conversion with figure rendering and classification. Key entry points are `make_book.py` and the installed command `bbook_maker`. The tool includes features like plan mode for selective translation, glossary support, resume, progress bars, and Docker support. It has extensive tests covering CLI contract, loaders, translators, pipeline stages, and documentation. External dependencies include LLM APIs, docling, pypdfium2, Pandoc, and various SDKs. The codebase demonstrates careful engineering decisions around capability degradation, session compaction, glossary learning, and structured output ladder. It is a well-maintained application with active development and low technical drift.