$ show Future-House/paper-qa
PaperQA2 is an agentic retrieval-augmented generation (RAG) system for scientific literature. It answers questions by searching, indexing, and analyzing PDFs, text documents, and clinical trials using a multi-step agent workflow. The system integrates with multiple scholarly metadata providers (Crossref, Semantic Scholar, OpenAlex, Unpaywall) for DOI resolution, citation counts, retraction detection, and journal quality scoring. It supports multiple LLM backends via LiteLLM, pluggable PDF readers (pypdf, PyMuPDF, Docling, Nemotron), and vector stores (Numpy, Qdrant). The architecture is a Python library with a CLI (`pqa`), featuring a hierarchical Pydantic settings system, an agent framework with tool selection and memory, and a plugin-based document parsing pipeline. While primarily an application, it is research-backed with published benchmarks (LitQA2, LFRQA) and includes config presets for different quality/performance trade-offs.
The LLM identified 4 files worth highlighting for reuse:
DocMetadataClient composes multiple scholarly API providers (Crossref, Semantic Scholar, OpenAlex, Unpaywall) and post-processors (journal quality, retractions) into a configurable metadata enrichment pipeline. It demonstrates clean separation of concerns, extensibility via provider/post-processor registration, and robust error handling with retries. This pattern is reusable for any project that needs to aggregate and enrich data from multiple external sources.
The Settings class provides a hierarchical Pydantic-based configuration system with nested sub-settings (AgentSettings, IndexSettings, ParsingSettings, PromptSettings, AnswerSettings). It supports named presets loaded from JSON files, CLI override via a custom source, and JSON serialization. This pattern is reusable for any complex application that needs flexible, layered configuration with validation and serialization.
The read_doc function and PDFParserFn protocol define a plugin architecture for document parsing. Backends (PDF, text, office, code) are registered via optional dependencies and selected at runtime. The protocol ensures a consistent interface (ParsedText) while allowing different implementations. This pattern is reusable for any system that needs to support multiple file formats with pluggable parsers.
The NamedTool base class provides a clean abstraction for agent tools with tool name registration, concurrency safety flags, and callback hooks. Concrete tools (PaperSearch, GatherEvidence, GenerateAnswer, ClinicalTrialsSearch) implement a consistent interface. This pattern is reusable for any agent framework that needs a modular, extensible tool system.