CacheControl is a Python library that adds HTTP caching to the popular `requests` library. It ports the caching algorithms from httplib2, providing a transparent caching layer for HTTP sessions. The library intercepts HTTP requests and responses, caching responses based on standard HTTP caching headers (Cache-Control, Expires, ETag) and optional heuristic rules. It supports multiple cache backends including in-memory dictionary, file-based cache (with separate body storage for memory efficiency), and Redis. The architecture is modular: a `CacheController` implements the core caching logic (freshness calculation, conditional requests, cache invalidation), while `CacheControlAdapter` integrates with requests' transport adapter system. The library also provides a CLI tool (`doesitcache`) for testing caching behavior. It is well-tested, type-annotated, and follows modern Python packaging practices.
omni-article-markdown is a CLI tool and library that converts web articles from various sources into clean Markdown format. It employs a modular pipeline architecture: Reader fetches HTML (using requests or Playwright for JavaScript-heavy sites), Extractor isolates article content by removing boilerplate and ads, and Parser converts the cleaned HTML to Markdown with support for code blocks, tables, math equations, and GitHub Gists. The system uses a plugin-based architecture where site-specific readers and extractors are automatically discovered and loaded, allowing easy extension for new websites. Key features include SSL verification control, cookie persistence for sites requiring authentication (like Zhihu), and flexible output options (stdout or file saving).
This repository provides 'Diagrams', a Python library that lets you define cloud system architecture diagrams as Python code. It renders diagrams using Graphviz, supporting a wide range of cloud providers (AWS, Azure, GCP, Kubernetes, etc.) through auto-generated node classes. The library includes a CLI tool to execute diagram scripts and a web-based playground (React/TypeScript) that runs the library in-browser via Pyodide, allowing interactive editing and export. The architecture is a library with a CLI entry point, plus a separate SPA playground. Key entry points are the `diagrams` CLI command and the `Diagram`/`Node` classes. External dependencies include Graphviz (system binary), Jinja2, and for the playground: Pyodide, @hpcc-js/wasm-graphviz, React, CodeMirror. The codebase is well-structured with a code generation pipeline for provider nodes, comprehensive testing, and a polished user experience.
BabelDOC is a production-grade PDF translation library and CLI tool that translates documents while preserving original layout, fonts, formulas, and styles. Its core architecture revolves around a custom Intermediate Representation (IL) that decouples PDF parsing from rendering, enabling a modular pipeline of parsing, layout analysis, LLM-based translation, typesetting, and PDF generation. The system supports complex PDF features including CJK text, Type3 fonts, tables, and formulas, and offers both monolingual and dual-language output modes. The translation pipeline is driven by an OpenAI-compatible LLM, with sophisticated prompt engineering for context-aware paragraph translation, glossary integration, and formula/style preservation. Layout analysis uses a DocLayout-YOLO ONNX model for detecting text blocks, tables, and figures. The codebase features a dual parser architecture, retaining a modified pdfminer for compatibility while developing a new native parser with improved font handling and security (CMap pickle deserialization hardening). BabelDOC demonstrates strong software engineering with several genuinely reusable components, including its typed IL data model, a priority thread pool executor, cross-platform memory monitoring utilities, and a high-performance glossary matching system using Hyperscan. The project is actively maintained with frequent releases, comprehensive CI/CD, and responsive security patching. While its complexity and dual-parser architecture present maintenance challenges, the overall code quality, architectural decisions, and reusable patterns make it a valuable reference for PDF processing and LLM-integrated document workflows.
grex is a regular expression generator that creates regex patterns from user-provided test cases. It implements DFA construction, Hopcroft's minimization algorithm, and Brzozowski's algebraic method to produce Perl-compatible regexes. The system serves dual purposes as a command-line utility and a library with bindings for Rust, Python, and WebAssembly. Key features include Unicode grapheme handling, configurable character class conversions (digits/whitespace/word), case insensitivity, repetition detection, and syntax highlighting. The architecture separates core regex building logic from CLI, Python, and WASM extension points, enabling reuse across multiple contexts.
This is a Python library for technical analysis of financial markets based on 'Chan Theory' (缠论). It computes complex structural elements such as fractals, strokes (bi), line segments, trading ranges (zhongshu), and buy/sell points from candlestick data. The library supports multiple data sources (BaoStock, Akshare, CSV, CCXT), multi-level analysis, customizable algorithms, and plotting via Matplotlib. The code is well-structured with clear separation of concerns, extensive use of caching for performance, and a flexible configuration system. It includes a GUI scanner application for A-shares and demo strategies. The library is designed for both research and production use, with incremental calculation capabilities for real-time updates. While the open-source version omits trading engine and ML features described in the full version, the core analytical engine is robust and reusable.
pymatgen (Python Materials Genomics) is a robust, open-source Python library for materials analysis, primarily used for high-throughput computational materials science. It provides core object representations for crystal structures, molecules, and computational outputs (VASP, Q-Chem, Gaussian, etc.), and extends into analysis modules for phase diagrams, battery electrodes, coordination environments, magnetism, elasticity, surface science, diffraction, and spectroscopy. The library powers the Materials Project database and includes CLI tools for structure manipulation, POTCAR generation, and plotting. It features extensive use of MSONable serialization, abstract base classes, and a correction scheme pattern for DFT energy adjustments. While heavily integrated with the Materials Project ecosystem, its modular design allows reuse of individual analysis components in other materials research pipelines.
PyMatting is a Python library for alpha matting: given an RGB image and a trimap that coarsely labels foreground, background, and unknown pixels, it estimates per-pixel foreground opacity (an alpha matte) and can extract a foreground cutout. The primary production entry points are the high-level `cutout()` function and the `pymatting` CLI, both of which pair closed-form alpha matting with multi-level foreground estimation. Below that, the library exposes a structured numerical toolbox: six alpha estimation algorithms (closed-form, KNN, large-kernel, learning-based, random-walk, and shared matting), multiple matting Laplacian constructions, foreground estimation on CPU/GPU, preconditioners, and a custom conjugate-gradient solver. The architecture is a modular Python package built on NumPy, SciPy, and Numba, with optional CuPy and PyOpenCL GPU backends for foreground estimation only.
dogpile.cache is a mature Python caching library that provides a generic caching API with a 'dogpile lock' mechanism to prevent thundering herd problems during cache regeneration. The library is structured around a central `CacheRegion` class that configures a pluggable backend (memory, memcached, redis, valkey, dbm, null) and exposes high-level operations like `get_or_create`, `set`, `get`, and `delete`. It also offers function decorators (`cache_on_arguments`, `cache_multi_on_arguments`) that automatically generate cache keys from function arguments and manage expiration and invalidation. The architecture follows a layered design: the `dogpile.lock` module implements the core dogpile lock (a mutex-based coordination pattern), the `dogpile.cache.api` module defines abstract base classes for backends and serialization, and concrete backends in `dogpile.cache.backends` implement the actual storage. The `CacheRegion` orchestrates key mangling, serialization, expiration, and invalidation strategies. External dependencies are minimal (decorator, stevedore, typing_extensions), with optional backends pulling in redis, valkey, pymemcache, etc. Key entry points include `make_region()` to create a region, `CacheRegion.configure()` to set up a backend, and `CacheRegion.get_or_create()` for atomic cache-and-generate operations. The library is well-tested with extensive unit and integration tests, uses modern Python features (type hints, ABCs, context managers), and follows a clean, maintainable code style. It is actively maintained as part of the SQLAlchemy project.
maxapi is an async Python SDK and bot framework for the MAX messenger platform. It provides a high-level client (`Bot`) for all MAX Bot API endpoints, an event-driven dispatcher with router hierarchy (`Dispatcher`/`Router`), composable filters (magic filter `F`, `Command`, `StateFilter`, etc.), middleware chains, a finite state machine (FSM) with multiple backends (memory, Redis), and webhook integrations for aiohttp, FastAPI, and Litestar. The library is heavily inspired by aiogram but adapted to MAX API specifics such as attachment types, callback payloads, and deep linking. It includes extensive utility code for formatting (HTML/Markdown), inline keyboard building, VCF parsing, and time helpers. The codebase is well-structured with Pydantic v2 models for all API entities, rigorous retry logic, and a comprehensive test suite covering unit, integration, and webhook scenarios. While not a standalone application, it serves as a foundational component for building production-ready MAX bots.
LangExtract is a Python library for extracting structured information (entities, relationships) from unstructured text using large language models (LLMs). It provides a production-grade pipeline: chunking long documents, building prompts with few-shot examples, invoking LLMs (Gemini, OpenAI, Ollama, or custom providers), parsing and fuzzy-aligning the output to source text, and generating interactive HTML visualizations. The architecture is modular with a plugin system for providers, a schema system for output constraints, and a resolver for alignment. It includes comprehensive tests, benchmarks, and CI/CD. The library is actively maintained with a clear deprecation path for v2.0.0. Notable strengths are the provider plugin system, fuzzy alignment algorithms, and format handling. Weaknesses include some deprecation debt and complex alignment code.
jc (JSON Convert) is a mature Python library and CLI tool that converts the output of over 150 command-line tools, system files, and common data formats into structured JSON. It operates both as a piped command-line utility (`jc <command>`) and as a programmable Python module (`jc.parse('parser_name', data)`), enabling seamless integration with jq, scripting, and automation pipelines. The architecture follows a consistent parser pattern: each parser module defines an `info` metadata class and a `parse(data, raw, quiet)` function, with optional `_process()` for type coercion. Streaming parsers (suffixed `_s`) use generator-based patterns for real-time output. The library includes vendored dependencies for ASN.1, plist, EDID, and TOML parsing, and supports cross-platform execution across Linux, macOS, FreeBSD, and Windows. The primary data flow is stdin or file input → parser module → structured dict/list output (JSON or YAML). Key entry points include the CLI (`jc/cli.py`), the library API (`jc/lib.py`), and individual parser modules under `jc/parsers/`. The project maintains an extensive test suite with fixtures captured from multiple OS versions, ensuring robustness against command output variations. It also provides shell completions, a plugin system for custom parsers, and documentation generation scripts. While the codebase is well-engineered with clean abstractions, the sheer number of parsers and reliance on vendored libraries introduce moderate maintenance surface area.
KiKit is a Python CLI tool, library, and KiCAD plugin for automating PCB panelization and fabrication output. It takes KiCAD board files (S-expression format) and produces panelized PCBs with v-cuts, mouse bites, tooling holes, fiducials, and frames. It also exports manufacturing files (Gerbers, drill, assembly data) tailored to popular fab houses (JLCPCB, PCBWay, OSH Park), generates 3D-printable solder paste stencils, performs DRC checks, and creates presentation pages for boards. The architecture is modular: core geometric operations use Shapely, interval algebra, and a custom S-expression parser; an extensible plugin system allows custom layout, tab, and cut strategies. The tool integrates deeply with KiCAD’s pcbnew API and supports multiple KiCAD versions (6–9) through version-specific adapters. While primarily a production tool for hardware engineers, its clean separation of parsing, geometry, and CLI makes several components reusable in other PCB-adjacent projects.
Rivalcfg is a Python 3 library and command-line tool for configuring SteelSeries gaming mice (Rival, Sensei, Prime, Aerox, Kana, Kinzu, etc.) on Linux, macOS, and Windows. It communicates with devices via HID (using hidapi) using reverse-engineered HID report protocols. The system provides a clean device profile dictionary schema for each mouse, a plugin-like handler system for different value types (choice, range, RGB color, button mapping, gradients, etc.), and a CLI that dynamically generates argparse options from the device profiles. It also includes persistent settings storage in JSON, udev rule management for Linux, and test doubles (FakeDevice, FakeMouseSettings) for testing without physical hardware. The project is community-driven, with new devices added frequently through reverse engineering.
Certifi is a minimal Python library that provides Mozilla's curated collection of root CA certificates for SSL/TLS verification. It exposes two functions: `where()` returns the filesystem path to the bundled `cacert.pem` file, and `contents()` returns its contents as a string. The library handles both modern Python 3.11+ importlib.resources API and older versions, with careful management of context managers and atexit cleanup to support zipimport scenarios. It is a dependency of many popular Python packages (e.g., requests) and is automatically updated daily via a GitHub Actions workflow that fetches the latest Mozilla CA bundle. The codebase is small, well-tested, and demonstrates high craftsmanship in resource management and backward compatibility.
Awkward Array is a Python library for manipulating nested, ragged, and mixed-type data structures that do not fit naturally into NumPy's rectangular array model. It is primarily a software library, not a data-research pipeline, though it contains research-oriented studies and benchmarks. The architecture is a Python front-end backed by compiled C++ kernels, with optional integrations for PyArrow, JAX, CuPy, Numba, CUDA, and ROOT. The core data model is built around composable layouts, forms, and types, and operations are dispatched through a high-level function decorator that separates public API from backend-specific implementations.
traffic is a Python library and CLI tool for processing, analyzing, and visualizing air traffic data. It ingests trajectory data from multiple sources including OpenSky Network (via REST/Trino), FlightRadar24, Eurocontrol (DDR, AIXM), FAA, and local ADS-B decoders. The core abstractions are `Flight` (single trajectory) and `Traffic` (collection) built on pandas DataFrames, with extensive methods for cleaning, resampling, filtering, geometric operations, navigation event detection (landing, go-around, holding patterns), performance estimation (fuel flow, emissions), and clustering. The library provides a declarative lazy evaluation pipeline (`LazyTraffic`) that chains operations before a single `.eval()` call, enabling efficient parallel processing. It also offers visualization backends for Matplotlib, Altair, Leaflet, and Plotly, and includes a Vue.js/Deck.gl plugin (Tangram) for real-time landing alignment in a web dashboard.
python-json-patch is a Python library implementing the JSON Patch format (RFC 6902) for applying operations like add, remove, replace, move, copy, and test to JSON documents. It includes a smart diff algorithm to generate minimal patches by comparing two JSON objects, and provides command-line utilities (jsondiff and jsonpatch) for file-based operations. The core logic resides in jsonpatch.py, which is self-contained aside from the jsonpointer dependency, and handles edge cases such as duplicate keys via a custom multidict function. The library supports Python 2.7 and 3.7+ (though its setup.py contains an version specifier error excluding 3.4-3.6).
The sentinel2-cloud-detector repository provides a Python package (s2cloudless) for detecting clouds in Sentinel-2 satellite imagery. It implements a pixel-based cloud detector using a pre-trained LightGBM model that takes Sentinel-2 spectral bands as input and outputs cloud probability maps or binary masks with optional post-processing (convolution and dilation). The package is designed as a reusable library for integration into remote sensing workflows. The architecture consists of three core modules: cloud_detector.py (containing the main S2PixelCloudDetector class), pixel_classifier.py (a generic wrapper for applying pixel-wise classifiers over image data), and utils.py (helper functions for data loading and image processing operations). The PixelClassifier class is particularly notable for its reusability, as it decouples the classification logic from the specific model and can work with any classifier adhering to the scikit-learn API (e.g., scikit-learn models, LightGBM). The package includes comprehensive unit tests, CI/CD pipelines for multiple Python versions, and follows modern packaging standards for distribution via PyPI and Conda. The package actively maintains compatibility with current dependencies (lightgbm, numpy, opencv, sentinelhub) and avoids deprecated APIs. It demonstrates strong software engineering practices through clear separation of concerns, type hints, and comprehensive documentation. While the S2PixelCloudDetector class is tailored to the specific cloud detection use case, the underlying PixelClassifier utility represents a broadly applicable pattern for pixel-wise image classification tasks.
search-engine-parser is a Python library and CLI tool designed to scrape search engines for structured results. It provides a base class for search engine implementations with specific parsers for engines like Google, Bing, Yahoo, DuckDuckGo, and others. The tool supports both synchronous and asynchronous HTTP requests, includes caching to avoid repeated requests, and uses user-agent rotation to prevent blocking. Key features include a command-line interface (pysearch) for terminal use and a programmatic API for integration into other projects, returning results as SearchItem objects with convenient access to titles, links, and descriptions.