$ show eLifePathways/sciencebeam-parser
ScienceBeam Parser is a scientific document parsing library and service that converts PDF/DOCX documents into structured formats (TEI/JATS XML) using machine learning models. It provides a FastAPI-based web service with GROBID-compatible endpoints, a CLI for training data generation, and a modular pipeline of models for segmentation, header, citation, reference extraction, etc. The system includes benchmarking tools for performance evaluation against GROBID baselines and training utilities for model improvement. While it contains research-oriented components (benchmarking, training), its primary character is as a deployable application/library for document processing.
The LLM identified 4 files worth highlighting for reuse:
Provides LazyLoaded and Preloadable classes for thread-safe, transparent lazy initialization of expensive resources (e.g., ML models). Zero dependencies beyond standard library, easily extractable, and demonstrates skill in managing initialization state and access patterns.
Integrates OpenTelemetry for tracing LLM API calls and provides contextvar-based usage tracking for token/cost metrics. Non-intrusive design separates telemetry concerns from business logic, showing expertise in adding observability to LLM interactions in production services.
Compares feature vectors from two models (e.g., parser vs. GROBID) to highlight token-level differences. Generic diff logic with clear output formatting, useful for debugging model behavior and adaptable to any feature-based model comparison task.
Factory pattern for creating TEI elements from semantic content (sections, paragraphs, citations, etc.). Isolated from parser logic, reusable for other XML generation tasks, and demonstrates skill in mapping semantic structures to XML with proper handling of attributes and nested elements.