Qlib is an open-source, AI-oriented quantitative investment platform developed by Microsoft. It covers the full pipeline for quantitative research and trading: data ingestion and feature engineering (Alpha158, Alpha360, high-frequency), model training (traditional ML like LightGBM/XGBoost, deep learning in PyTorch including LSTM, GRU, Transformer, GATs, and reinforcement learning), backtesting with realistic costs and slippage, portfolio optimization, and online serving for production-like deployment. The platform is highly modular and config-driven, with a core library (qlib) and a separate contrib layer for community extensions. It includes a CLI (qrun) for running full workflows from YAML configs, extensive experiment tracking via MLflow, and a rich set of evaluation metrics. While primarily a library/application, it also functions as a research benchmark suite with many example models and data collectors for global markets. The codebase is active and well-maintained, though some components (e.g., TFT model, older RL code) show signs of technical drift due to dependency version locks and deprecated patterns.
RealtimeSTT is a production-grade real-time speech-to-text library and server. The core library provides `AudioToTextRecorder`, a facade that orchestrates microphone capture, voice activity detection (VAD), wake-word detection, and multiple ASR backends (faster-whisper, sherpa-onnx, whisper.cpp, etc.) for low-latency realtime and final transcription. It features a dual-lane architecture combining an ultrafast speculative model with a slower accurate model, merged via a sticky alignment algorithm, and a text stabilizer that commits only evidence-backed partials. The server component (`stt-server-production`) is a FastAPI/WebSocket application with a versioned protocol (`realtimestt.remote.v1`) supporting per-session ordered event delivery, fair inference queues, preview snapshots, resume, and HTTP transcription. External dependencies include PyTorch, torchaudio, faster-whisper, sherpa-onnx, and optional engines. The system is designed for voice assistants, dictation, and multi-user ASR services.
EconML is a Python library for estimating heterogeneous treatment effects (CATE) from observational and experimental data using machine learning. It implements state-of-the-art causal inference methods including Double/Debiased Machine Learning (DML), Generalized Random Forests (GRF), Orthogonal Random Forests, Doubly Robust Learners, Instrumental Variable estimators (DMLIV, DRIV, OrthoIV), and Meta-Learners (S, T, X, DA). The library provides a unified scikit-learn-compatible API across all estimators with fit, effect, marginal_effect, and inference methods (confidence intervals, p-values). It includes advanced features such as cross-fitting, sample weights, missing data support, federated learning, sensitivity analysis, and model selection via RScorer. The repository also contains extensive documentation, notebooks, Monte Carlo tests, and a Cython-optimized tree implementation for performance.
Twikit is an asynchronous Python library that provides a comprehensive, unofficial client for the Twitter (X) API. It allows users to perform a wide range of Twitter operations—such as posting tweets, searching for tweets and users, managing bookmarks and lists, sending direct messages, uploading media, and streaming real-time events—without requiring an official API key. The library achieves this by reverse-engineering Twitter's internal GraphQL and REST v1.1 endpoints, handling authentication via login flows or guest tokens, and implementing sophisticated anti-bot evasion techniques like client-side transaction ID generation (X-Client-Transaction-Id) and UI metrics obfuscation. The architecture separates concerns into GraphQL (gql) and REST (v11) client modules, with data models for tweets, users, communities, and more. It includes optional CAPTCHA solving and cookie persistence for session reuse. The codebase is well-structured, actively maintained, and demonstrates deep reverse-engineering skill, particularly in the `x_client_transaction` module that mimics Twitter's client-side telemetry.
This repository is the companion to Allen Downey's Think DSP book. It is primarily an educational Python library plus a collection of Jupyter notebooks, not a production service or a research data pipeline. The core deliverable is the `thinkdsp` package, a self-contained module that provides object-oriented primitives for representing periodic signals (`Signal`, `Sinusoid`, `Chirp`, `Noise`), sampling them into `Wave` objects, and transforming between time and frequency domains via FFT and DCT. The library also supports spectral arithmetic, filtering, convolution, autocorrelation, spectrogram generation, WAV file I/O, and basic audio synthesis. The notebooks in `nb/`, `split/`, and `examples/` walk through each chapter of the book, with solution notebooks and standalone solution scripts in `soln/`.
SALib is a Python library for global sensitivity analysis (GSA) of computational models. It implements a comprehensive suite of variance-based and moment-independent methods including Sobol', Morris, FAST, DGSM, PAWN, HDMR, RBD-FAST, and fractional factorial designs. The library follows a decoupled workflow: sample parameter space, evaluate the model, then analyze results. It provides both a programmatic API (import SALib) and a command-line interface (salib sample/analyze). The architecture is modular: sampling modules in SALib.sample, analysis modules in SALib.analyze, with shared utilities in SALib.util. A high-level ProblemSpec class enables fluent method chaining (sample → evaluate → analyze → plot). Results are returned as ResultDict objects with to_df() and plot() methods for integration with pandas and matplotlib. External dependencies are numpy, scipy, matplotlib, pandas, and optionally multiprocess/pathos for parallel execution. The library is used in environmental modeling, energy systems, and uncertainty quantification research, with over 1000 GitHub stars and a JOSS paper.
Tax-Calculator is a mature, open-source microsimulation library for static (and simple behavioral) analysis of the US federal individual income and payroll tax system. It provides a Python API and a CLI tool (`tc`) that allow users to define policy reforms via JSON parameter files, apply them to microdata samples (CPS, PUF, or TMD), and produce distributional tables, revenue estimates, marginal tax rates, and welfare measures. The core architecture separates policy parameters (`Policy`), taxpayer records (`Records`), and economic adjustments (`GrowFactors`, `GrowDiff`, `Consumption`) from the calculation engine (`Calculator`), which uses Numba JIT compilation for performance. While primarily an application/library with formal releases and CI/CD, it also bundles research data (CPS microdata, grow factors) and includes extensive cross-validation against NBER's TAXSIM-35 model. The codebase is well-structured, heavily tested, and actively maintained, with a clear modular design that supports both CLI and programmatic use.
yfinance is a mature Python library that provides programmatic access to Yahoo Finance market data. It is primarily an application/library, not a data research project, though it includes data repair and cleaning utilities. The library exposes a high-level API (Ticker, download, Search, Lookup, Market, Screener) that fetches historical prices, fundamentals, financial statements, holders, analysis, and more from Yahoo's various JSON endpoints. It handles Yahoo's quirks (cookie/crumb authentication, timezone issues, data errors) and includes a sophisticated price repair pipeline that detects and corrects common data anomalies like 100x unit mixups, bad stock splits, zero values, and dividend double-counting. The architecture is monolithic with modular scrapers and utilities. It has extensive test coverage (though currently failing in CI), active maintenance, and a large user base (25k+ stars). The codebase is well-structured with reusable utility functions, but some legacy artifacts (Travis CI, disabled pytest workflow) and heavy reliance on heuristic repair logic introduce medium technical drift.
dataset is a lightweight Python library that simplifies database operations by providing a dict-like interface on top of SQLAlchemy. It automatically creates tables and columns on insert, supports CRUD operations, transactions, and chunked bulk inserts/updates. The library is designed for small to medium-scale data tasks where full ORM overhead is unnecessary. It supports SQLite, PostgreSQL, and MySQL, and is fully type-annotated with mypy strict compliance. The codebase is well-organized into core modules (database, table, types, util, chunked) and has thorough test coverage across all supported backends. While the core abstraction is thin (intentionally), the implementation demonstrates solid engineering discipline with clean APIs, thread safety considerations, and proper handling of SQLAlchemy 2.x quirks.
Merlion is a mature Python library for time series intelligence, specializing in anomaly detection and forecasting. It provides a comprehensive, modular ecosystem including model zoo (classical, tree-based, deep learning, ensemble, AutoML), composable transforms, post-processing chains, evaluation frameworks, and a Dash-based interactive dashboard. The architecture follows a clean layered design: data loading via `ts_datasets`, core `TimeSeries` abstraction, `ModelBase` hierarchy with config-driven instantiation, invertible transforms, and pluggable post-rules. It integrates with Spark for distributed inference and provides benchmarks against standard datasets (NAB, M4, UCR). Though primarily a library (application), it includes research-oriented components such as state-of-the-art transformer architectures (Autoformer, Informer) and Bayesian change point detection, making it a hybrid tool suitable for both production and research.
PaCMAP is a production-ready Python library for dimensionality reduction and manifold learning, implementing the Pairwise Controlled Manifold Approximation Projection algorithm. The core package (source/pacmap/) provides a scikit-learn compatible API with BaseEstimator inheritance, enabling seamless integration into ML pipelines. The library supports multiple KNN backends (FAISS, Annoy, Voyager) through a unified abstraction layer, with heavy use of Numba JIT compilation for performance-critical operations like distance calculation and gradient updates. Research components (experiments/, evaluation/, data/) support algorithm validation against benchmarks like t-SNE and UMAP, but the primary deliverable is the reusable library itself. The codebase demonstrates strong engineering practices with comprehensive pytest coverage, model persistence mechanisms, and deterministic random state management across Numba functions.
vkbottle is an asynchronous Python framework for building VK (VKontakte) bots and interacting with the VK API. It provides both high-level abstractions (Bot, User, labelers, rules, middlewares, state dispensers) and low-level API clients with pluggable request/response validation, token rotation, and error handling. The framework is fully async, built on asyncio and aiohttp, and supports long polling, callback API, and dual mode (simultaneous polling and callback). It includes extensive tooling for keyboards, templates, uploaders, markdown-to-VK-format conversion, VKScript generation, event deduplication, and a waiter machine for awaiting future events. The architecture is modular with abstract base classes for nearly every component, enabling custom implementations. The codebase is well-structured with clear separation of concerns, extensive test coverage, and CI/CD pipelines. Deprecated components (LoopWrapper, Blueprints) are clearly marked, and the library is actively maintained with a mature version 4.11.0.
This repository provides a SQLAlchemy dialect for DuckDB, enabling DuckDB to be used as a database backend within SQLAlchemy's ORM and Core APIs. It maps DuckDB's type system to SQLAlchemy types, handles connection management (including in-memory and file-based databases), supports reflection (schema introspection), and integrates with DuckDB extensions. The dialect is built on top of the PostgreSQL psycopg2 dialect, customizing it for DuckDB's capabilities and limitations. The source code is well-structured, extensively tested, and actively maintained, with support for DuckDB versions 0.5.0 through 1.2.2 and SQLAlchemy 1.3 through 2.0.35.
Perception is a Python library for perceptual image and video hashing, developed by Thorn. It provides a unified interface for multiple hashing algorithms (e.g., pHash, dHash, PDQ, TMK), approximate nearest neighbor search via FAISS with graph-based clustering, and a benchmarking framework for evaluating hasher robustness under various transformations. The library is designed for production use cases like near-duplicate detection, content matching, and CSAM detection via integration with Thorn's Safer service. The codebase demonstrates strong engineering practices: clean abstractions, Cython acceleration, optional dependency management, and extensive test coverage. While primarily an application/library, it includes research-oriented benchmarking pipelines and evaluation datasets, making it a hybrid of production tooling and research infrastructure.
foureng is a Python library for quantitative finance: Fourier and characteristic-function based option pricing. It exposes unified dispatchers (`price_strip` and `price`) over a registry of characteristic-function models (BSM, Heston, VG, Kou, Bates, CGMY, NIG, 3/2, rough Heston, etc.) and numerical engines (COS, Carr-Madan FFT, FRFT, Lewis, Hilbert, PROJ, CTMC, SINC, SWIFT, contour, plus Monte Carlo baselines). The product layer covers European vanillas, Bermudan/American, barriers, Asians, lookbacks, cliquets, faders, variance/credit instruments, and multi-asset payoffs. It is a library, not a service or CLI: usage is through the Python API and Jupyter notebooks.
ScrapeGraphAI is a Python library that extracts structured data from websites and local documents using LLM-driven graph pipelines. Instead of a traditional web scraper with hardcoded selectors, users define extraction through natural-language prompts and optional Pydantic schemas; the library then executes a directed graph of nodes such as fetching, parsing, chunking, and answer generation. The public API is a set of graph classes (SmartScraperGraph, SearchGraph, JSONScraperGraph, CSVScraperGraph, SpeechGraph, ScriptCreatorGraph, etc.) in scrapegraphai/graphs, each composing reusable nodes from scrapegraphai/nodes and running them on a BaseGraph execution engine. It is consumed programmatically in Python and supports many LLM providers (OpenAI, Ollama, Groq, Mistral, Anthropic, DeepSeek, xAI, NVIDIA, MiniMax) and multiple fetching backends (Playwright, Selenium, Chromium, BrowserBase, Scrape.do). The architecture is a lightweight DAG framework: nodes have typed inputs/outputs, can branch conditionally, iterate, merge results, and accumulate cost/token metadata; BaseGraph walks the graph, handles state, and collects execution information. The repository also contains document loaders, model token/cost registries, prompt templates, telemetry, proxy rotation, screenshot-scraping utilities, a GraphBuilder that constructs graphs from natural language, and integrations such as Burr and Indexify. While the README aggressively promotes a commercial hosted API at scrapegraphai.com, the repository itself is a self-contained open-source library with examples and tests. The codebase is generally well structured and modular, with clear separation between graphs, nodes, prompts, helpers, utils, and models. It shows signs of rapid growth: some typos, duplicated test fixtures, hardcoded model pricing tables, a Python version mismatch between pyproject and Dockerfile, and dependency fragility around LangChain and optional browser/OCR packages. These issues are manageable but give the codebase a medium technical drift. Overall it is an active, useful library for LLM-based extraction, and several utilities are good candidates for reuse outside the project.
Activeloop's DeepLake is a production-grade, multi-vector database engine for AI data. At its core, it is a columnar, chunked, compressed storage system for tensors (multi-dimensional arrays) with native support for embeddings, images, video, text, and other complex AI data types. It exposes a Python library for dataset management, ingestion (COCO, Labelbox, CSV, Parquet), version control, and integration with PyTorch/TensorFlow/MMDetection. A high-performance C++ backend provides lazy NumPy-like array operations (nd::array), a composable async framework (async::promise), and a Tensor Query Language (TQL). The most distinctive component is a PostgreSQL extension (pg_deeplake) that allows SQL access to DeepLake datasets via custom table and index access methods, supporting vector similarity search (cosine, BM25) and hybrid search. Query execution is delegated to DuckDB through a custom scan node, enabling efficient analytical queries. The extension also includes WAL-based DDL replication across instances. This is primarily an application (a database system), with substantial research-oriented features such as medical image formats, compression codecs, and TPC-H benchmarking.
DoWhy is a mature Python library for causal inference that implements a systematic four-step workflow: model (encode causal graph), identify (derive estimand), estimate (compute causal effect), and refute (validate robustness). The library supports two major frameworks: (1) classical potential-outcomes-based estimation with ATE/ATT/CATE via backdoor, frontdoor, instrumental variables, and mediation analysis, leveraging multiple estimators including linear regression, propensity score methods, doubly robust, instrumental variable, regression discontinuity, and integrations with EconML and CausalML; (2) Graphical Causal Model (GCM) based inference for what-if analysis, counterfactual sampling, distribution change attribution, arrow strength, intrinsic causal influence, anomaly attribution, and root cause analysis. The architecture is modular with abstract base classes for estimators, refuters, interpreters, and GCM components. The main entry point is the CausalModel class, which orchestrates identification, estimation, and refutation via a consistent API. Key dependencies include scikit-learn, statsmodels, networkx, sympy, pandas, numpy, and optional integrations with Torch, EconML, Causal-learn, and TabPFN. The library is production-ready with comprehensive test suites, CI/CD, and extensive documentation, and is actively maintained with a large contributor base.
pycountry is a Python library that provides programmatic access to ISO standard data including countries (ISO 3166-1), subdivisions (ISO 3166-2), historic countries (ISO 3166-3), currencies (ISO 4217), languages (ISO 639-3), language families (ISO 639-5), and scripts (ISO 15924). It packages JSON databases and compiled GNU gettext translation files (`.mo`) for dozens of locales, sourced from the Debian iso-codes project. The library is a pure wrapper—no data transformation or analysis is performed—and explicitly rejects data contributions. Its core API exposes objects like `pycountry.countries`, `pycountry.languages`, and `pycountry.currencies` with methods for exact lookups (`get`, `lookup`), fuzzy search (`search_fuzzy`), and dynamic entry modification (`add_entry`, `remove_entry`). Translation support is provided via locale-aware gettext catalogs, allowing localized name retrieval for supported languages. The architecture is a monolith library with minimal external dependencies (only standard library modules). Single Python package, no web server or CLI. Data is loaded lazily and cached. The test suite covers all database objects and edge cases.
kagglehub is the official Python library for interacting with Kaggle resources. It provides a unified API to download and upload models, datasets, competition data, and notebook outputs, with automatic handling of caching, authentication (including Colab secrets and JWT), and environment-specific resolution (direct HTTP, Kaggle notebook cache, Colab mount). The library uses a multi-resolver architecture where each resource type (model, dataset, etc.) is resolved by a chain of resolvers that are tried in priority order (Colab cache > Kaggle cache > HTTP). It also includes dataset loading adapters for pandas, polars, and Hugging Face Datasets, and supports Kaggle Packages (executable notebooks that export Python modules). The codebase is well-structured, extensively tested (with unit and integration tests using Flask stubs), and follows modern Python practices (type hints, linting, hatch build system).