The openai-python repository is the official Python SDK for interacting with OpenAI's API services. It provides both synchronous and asynchronous clients for accessing OpenAI's REST and WebSocket APIs, including chat completions, embeddings, image generation, audio processing, fine-tuning, vector stores, and admin organization management. The library is generated from OpenAPI specifications using Castiron, ensuring strong type safety via Pydantic models and TypedDict for request/response objects. Key features include streaming support, webhook verification, workload identity authentication, and modular resource organization. The SDK handles authentication, retry logic, pagination, and error handling transparently, making it suitable for production AI-integrated applications.
MechanicalSoup is a Python library for automating interaction with websites. It simulates a web browser by automatically storing and sending cookies, following redirects, and submitting forms. It does not execute JavaScript, making it a lightweight alternative to Selenium for sites that rely on server-side logic. The library is built on top of Requests for HTTP sessions and BeautifulSoup for HTML parsing and manipulation. The architecture provides two main classes: Browser (low-level, stateless) and StatefulBrowser (high-level, stateful). StatefulBrowser is the recommended interface, maintaining the current page, URL, and selected form as internal state. Users can open URLs, follow links, select forms, fill in fields, and submit. The Form class handles form manipulation including text inputs, checkboxes, radio buttons, select dropdowns, textareas, and file uploads. It also correctly implements HTML form submission semantics such as disabled controls, fieldset disabling, and multipart encoding. Key entry points include StatefulBrowser.open(), select_form(), submit_selected(), follow_link(), and download_link(). The library depends on requests, beautifulsoup4, and lxml. It is designed as a reusable library, not a standalone application, and is typically installed via pip and imported into user scripts. The codebase is well-tested with 100% coverage, has comprehensive documentation, and follows semantic versioning.
This repository is `imroc/req`, a feature-rich, production-grade HTTP client library for Go (version 3). It provides a high-level fluent API for building and executing HTTP requests with support for HTTP/1.1, HTTP/2, and HTTP/3 (via QUIC). Key capabilities include connection pooling, automatic retries with configurable backoff, redirect policies, request/response middleware, TLS fingerprint impersonation (mimicking Chrome, Firefox, Safari), SOCKS4/4a/5 proxying, compression (gzip, deflate, brotli, zstd), response body auto-decode (charset detection), file upload/download with progress callbacks, parallel downloading, and extensive debugging via request/response dump and trace. The library is built on top of Go's `net/http` transport but extends it significantly with custom implementations for HTTP/2 frames, HTTP/3, and SOCKS. It is designed for both simple API calls and complex integrations, as demonstrated by included examples for OpenTelemetry tracing and GitHub API usage. The codebase is well-tested and actively maintained, targeting recent Go versions (1.25+). While primarily an HTTP client library, it includes self-contained utility packages for charset detection, SOCKS proxy, and compression that could be reused independently.
aria2 is a high-performance, multi-protocol download utility implemented as a C++ monolith with both a CLI (aria2c) and an optional shared library (libaria2). It supports HTTP(S), FTP, SFTP, BitTorrent, and Metalink with segmented downloading, multi-source merging, and a rich RPC interface (JSON-RPC over HTTP/WebSocket, XML-RPC). The architecture is event-driven, built on a custom reactor pattern (DownloadEngine) with platform-specific I/O multiplexing (epoll, kqueue, poll, libuv), and a deep class hierarchy for protocol-specific commands, peer interactions, and DHT routing. External dependencies are abstracted behind internal interfaces (e.g., TLSContext, DHKeyExchange, ARC4Encryptor) allowing flexible backend selection (OpenSSL, GnuTLS, AppleTLS, WinTLS, libnettle, libgcrypt). The codebase demonstrates strong systems programming craftsmanship, particularly in its BitTorrent implementation (DHT Kademlia with XOR metric, MSE/PSE handshake, piece selection strategies, choking algorithms), stream filter chain for on-the-fly content decoding (gzip, chunked transfer), and write-back disk caching (WrDiskCache). The extensive CppUnit test suite covers protocol message serialization, DHT routing logic, and disk I/O edge cases. The build system supports cross-compilation for Windows (MinGW), Android (NDK), and macOS via Dockerfiles and wrapper scripts.
Colly is a mature, high-performance web scraping framework for Go, providing a clean, callback-based API for crawling and extracting structured data from HTML and XML. It handles HTTP requests, session management, caching, robots.txt compliance, proxy rotation, rate limiting, and parallel scraping out of the box. The library is built around a Collector object that orchestrates configurable callbacks (OnRequest, OnResponse, OnHTML, OnXML, OnError, OnScraped) and supports chaining, context propagation, and request queueing for large-scale scraping jobs. The codebase is well-structured, thoroughly tested, and follows Go idioms, making it a reliable choice for production scraping tasks. It is not a data research project; it is a general-purpose scraping library.
This is the official Google API Client Library for PHP, providing a unified interface to call Google APIs such as Drive, YouTube, Books, and many others. It handles authentication (API keys, OAuth2 web flow, service accounts), HTTP requests via Guzzle, batching, media uploads (simple, multipart, resumable), retries with exponential backoff, caching, and model serialization. The library is in maintenance mode: critical bugs and security issues are addressed, but no new features are added. It is a mature, well-tested library with extensive documentation and examples. The architecture centers around a `Google\Client` object that manages authentication and HTTP, and generated service classes (from `google/apiclient-services`) that map to specific Google APIs. The codebase is well-structured, uses modern PHP (8.0+), and follows PSR standards. Key reusable components include a standalone URI Template parser and a generic retry runner with exponential backoff.
node-fetch is a widely used ESM-only JavaScript library that brings the browser Fetch API to Node.js. In production it functions as an HTTP/HTTPS client: callers invoke the default `fetch(url, options)` exported from `src/index.js` and receive a `Response` backed by native Node.js Readable streams. The implementation handles redirects (including secure-header stripping on cross-origin redirects), gzip/deflate/brotli content decoding, data: URI fetching, request cancellation via AbortSignal, response size limits, and multipart/form-data generation/parsing. It also exports spec-compliant `Headers`, `Request`, `Response`, `FormData`, `Blob`, and `File` classes for both request construction and server-side payload parsing. Architecturally, the code is a modular library, not a service: `src/body.js` provides the shared Body mixin; `src/request.js` and `src/response.js` build on it; `src/headers.js` implements the Web IDL Headers interface; `src/index.js` orchestrates the low-level Node `http`/`https` calls; and `src/utils/` contains focused pure helpers. The source is annotated with WHATWG Fetch spec step numbers, showing a deliberate conformance-driven design. Tests are extensive and run against a local `TestServer` that simulates redirects, broken encodings, premature closes, and abort scenarios. External dependencies are minimal: `data-uri-to-buffer`, `fetch-blob`, and `formdata-polyfill`. The package declares Node `^12.20.0 || ^14.13.1 || >=16.0.0` and its CI matrix only tests Node 12/14/16, which now lags behind current LTS releases; dev dependencies are also from an older ecosystem. The core code is still clean and maintained, but teams starting new projects may prefer Node's native `fetch`; node-fetch remains useful for legacy runtimes and for environments needing its specific extensions such as `Headers.raw()` and custom agents.
govcr is a Go library that records and replays HTTP/HTTPS interactions for testing purposes, acting as an HTTP mock. It wraps an http.Client to intercept requests, either playing back recorded responses from cassette files (JSON) or making live requests and recording them. The library supports various modes (normal, live-only, offline, read-only), cassette encryption (AES-GCM, ChaCha20Poly1305), multiple storage backends (local filesystem, AWS S3), flexible request matching, and request/response transformation via mutators. The architecture centers around a custom http.RoundTripper implementation that integrates cassette storage, request matching, mutation systems, and encryption layers.
curlconverter is a command-line tool and library that converts curl commands into equivalent code snippets in over 30 programming languages and HTTP libraries. It features a modular architecture with a tree-sitter based Bash parser for robust curl command parsing, language-specific generators for target code production, and extensive test coverage. The tool handles complex curl features including authentication, headers, data uploads, multipart forms, proxies, and SSL options, producing runnable code in languages ranging from Python and JavaScript to Rust, Go, and specialized tools like Ansible and HTTPie.
This repository provides a set of PHP interfaces (contracts) for HTTP client operations, extracted from the Symfony framework. It defines the core abstractions: HttpClientInterface for issuing requests, ResponseInterface for lazy response handling, ChunkInterface for streaming response chunks, and ResponseStreamInterface for iterating over chunks from multiple responses. A full hierarchy of exception interfaces (Transport, Client, Server, Redirection, Timeout, Decoding) is included, along with a reference test suite (HttpClientTestCase) that validates any implementation against a set of comprehensive scenarios. The package contains no concrete implementations; it is purely a contract to ensure interoperability among different HTTP client libraries. The code is clean, well-documented, and leverages modern PHP 8.1 features like mixed types and static return types. It is actively maintained with zero open issues and follows semantic versioning as part of the Symfony contracts.
autocannon is a high-performance HTTP/1.1 benchmarking tool written in Node.js, inspired by wrk and wrk2. It operates as both a CLI tool and a programmable library. The architecture is centered around a run loop that manages multiple concurrent HTTP clients, each capable of pipelining requests. Key components include a request builder that constructs raw HTTP request buffers, a pipelined request queue for handling out-of-order responses, and a histogram-based statistics engine using hdr-histogram-js with WebAssembly for efficient percentile calculations. The tool supports HTTPS, Unix sockets, multipart forms, HAR file replay, worker threads for parallel execution, rate limiting with coordinated omission correction, and extensive customization via setup functions and event hooks. The codebase is well-structured with clear separation of concerns, comprehensive test coverage, and active maintenance.
Goose is a high-performance, Rust-based load testing framework designed to simulate realistic HTTP/HTTPS user traffic. It functions as both a reusable library and a CLI tool, allowing developers to write load tests in Rust that are compiled into standalone binaries. The architecture relies on the Tokio async runtime for efficient concurrency, utilizing flume channels to decouple virtual user threads from the main metrics aggregation process. It supports complex test scenarios including weighted transactions, session data management, and custom protocol simulation (TCP, WebSocket, gRPC). The framework distinguishes itself through rigorous metric accuracy, specifically implementing mitigation strategies for Coordinated Omission (CO) to prevent misleading latency data during server stalls. It provides extensive reporting capabilities, generating HTML reports with ECharts visualizations, JSON metrics for CI/CD integration, and PDF reports via headless Chrome. Runtime control is available through Telnet and WebSocket interfaces, allowing test parameters to be adjusted dynamically without stopping the load generation. While the core library is stable and well-architected, the project exhibits signs of active but evolving development. The distributed testing feature (Gaggle) was removed and is currently being re-implemented, indicating ongoing architectural shifts. The project maintains a comprehensive documentation suite via mdBook but faces challenges in synchronizing documentation across multiple locations (README, CHANGELOG, mdBook, inline code). The AI-assisted development workflow relies on external APIs that may introduce dependency risks if those services change. For engineers evaluating reuse, Goose offers a robust foundation for performance testing with strong type safety guarantees in its client builder. The Coordinated Omission mitigation logic is a unique, high-value component for teams concerned with metric accuracy. However, users should be aware of the optional PDF dependency (Chrome) and the current instability of the distributed testing mode. The codebase is suitable for integration into CI/CD pipelines or as a library for custom performance validation tools.
zlFetch is a lightweight JavaScript library that wraps the native Fetch API with quality-of-life improvements. It automatically parses response bodies based on `Content-Type` (JSON, text, form-urlencoded, NDJSON, Blob), provides promise-based error handling that rejects on 4xx/5xx status codes, supports request aborting, and includes shorthand methods for GET, POST, PUT, PATCH, DELETE. The library also offers streaming capabilities (SSE, NDJSON, chunked transfer), an EventSource wrapper for server-sent events, query string helpers, authorization header helpers, and a `createZlFetch` factory for pre-configured instances. The code is modular, well-typed with JSDoc, and thoroughly tested across browser and Node.js environments using vitest.
Fuel is a Kotlin Multiplatform HTTP networking library built on kotlinx.coroutines, providing suspend-style HTTP verbs (GET, POST, PUT, PATCH, DELETE, HEAD, arbitrary methods) plus Server-Sent Events streaming across JVM, Apple (iOS/macOS), and WasmJS targets. The core fuel module uses an expect/actual split: a common HttpLoader interface and FuelBuilder are backed by OkHttp on JVM, NSURLSession on Apple, and the browser fetch API on WasmJS. A global Fuel object supports plugging in a custom loader or loader factory, and extension functions on String (httpGet(), httpPost(), etc.) provide concise request entry points. A FuelRouting interface lets applications model typed API endpoints as sealed classes. Four optional serialization modules — fuel-jackson-jvm, fuel-moshi-jvm, fuel-kotlinx-serialization, and fuel-forge-jvm — each add a small extension on HttpResponse (toJackson, toMoshi, toJson, toForge) that decodes the body and returns a Result from the companion Result library. The core response model is deliberately minimal: a mutable HttpResponse holding a status code, a kotlinx.io Source body stream, and headers. JVM request handling is semantically correct about HTTP (e.g., consulting OkHttp's HttpMethod.permitsRequestBody before attaching a body). The project is published as 3.0.0-alpha04, uses Kotlin 2.3 / Gradle 9.3 / OkHttp 5 alpha, and is actively maintained with CI, Renovate, Dependabot, and ktlint. JVM has solid unit coverage using MockWebServer for all verbs, routing, custom OkHttp configurations, and each serialization adapter. Apple and Wasm targets are implemented but largely untested outside manual samples, and the Apple SSE implementation is a naive full-body line splitter rather than true incremental streaming. Overall this is a clean, modern, well-organized library with a couple of genuinely reusable pieces (notably the Android-derived UriCodec port) and some pre-1.0 rough edges. This is a pure application/library repository, not a data project.
goex is a Go library that provides unified interfaces for cryptocurrency exchanges (OKX, Binance, Bitget). It abstracts exchange-specific API differences behind common interfaces for both public market data and private trading operations. The architecture features modular exchange-specific packages, shared utilities for HTTP clients, logging, and model definitions, and uses dependency injection for configuration. Key strengths include clean separation of concerns, proper implementation of cryptographic signing, and use of the functional options pattern for configuration.
Methanol is an extensible HTTP client library built on Java's java.net.http package. It provides core HTTP functionality enhanced with features like automatic decompression (gzip, deflate, Brotli), pluggable body adapters for JSON/XML/Protobuf, HTTP caching with multiple backends (memory, disk, Redis), request/response interceptors, retry mechanisms, progress tracking, and Kotlin DSL support. The library follows a modular architecture with separate modules for core functionality, format adapters, extensions, and testing. Key entry points include Methanol.Builder for client creation, MutableRequest for request building, and BodyAdapter for encoding/decoding. Data flows involve request interception, body encoding, HTTP exchange via java.net.http, response interception, body decoding, and optional caching/progress tracking. External dependencies vary by module: core requires only JDK 11+, while adapters depend on libraries like Jackson/Gson/Moshi, extensions on Redis Lettuce or native Brotli, and testing on JUnit5/MockWebServer.
unzip-http is a Python-based command-line tool and library for extracting specific files from a ZIP archive hosted over HTTP without downloading the entire archive. It achieves this by leveraging HTTP range requests to download only the necessary portions of the ZIP file, such as the central directory and the compressed data of individual files. The tool can list the contents of a remote ZIP or extract one or more files (with support for wildcards) to the local filesystem or to stdout. The architecture centers around the `RemoteZipFile` class, which handles the HTTP communication and ZIP file parsing, and the `ZipStream` class, which provides on-the-fly decompression of deflate-compressed data. The CLI entry point in `unzip_http.py` provides a user-friendly interface for common operations, while the same code can be imported and used as a library in other Python programs. The project relies solely on the `urllib3` library for HTTP operations. Two components stand out for their reusability and technical merit: the `ZipStream` class, which implements a streaming decompressor for deflate data that can be used in any context requiring incremental decompression, and the `ZipInfo.parse_extra` method, which robustly handles ZIP64 extra fields to support large files. The project is actively maintained, with recent updates supporting modern Python versions and improving robustness (e.g., better handling of missing Accept-Ranges headers).
The Parcle Python client is a library for interacting with the Parcle Memory API, which provides long-term memory for AI agents. It enables developers to store conversational data and files per user, and then retrieve relevant information through natural language queries that return synthesized answers with citations. The client handles all aspects of communication with the API, including authentication, request retries, file uploads, and processing of Server-Sent Events for search operations. The architecture is straightforward: a synchronous HTTP client built on top of `httpx`, with methods corresponding to each API endpoint. Key features include robust error handling via a well-defined exception hierarchy, automatic retry logic for transient failures, and flexible file upload support that accepts various input types while enforcing size limits. The search method uniquely handles SSE streams to deliver incremental responses, ensuring compatibility with the API's streaming behavior. While the library is purpose-built for the Parcle API, it exhibits several reusable patterns. The exception mapping system in `exceptions.py` can be adapted for any HTTP API, and the file upload handling functions in `client.py` demonstrate a robust approach to processing diverse file inputs. The codebase is actively maintained, has low technical debt, and includes comprehensive tests that mock HTTP interactions to ensure reliability.
This is a Spring Boot starter library that deeply integrates Square's Retrofit HTTP client with Spring Boot's auto-configuration, allowing developers to declare typed HTTP APIs via Java interfaces annotated with `@RetrofitClient`. The library provides a comprehensive set of features: automatic OkHttpClient management, a pluggable interceptor chain (global, network, path-match), configurable retry with exponential backoff and jitter, circuit breaking (via Sentinel or Resilience4j), logging strategies, Micrometer metrics, custom error decoding, and connection pooling. It also supports Spring Boot Actuator for runtime introspection, GraalVM native image/AOT compilation, and reactive call adapters (RxJava 2/3, Reactor). The architecture is modular with clear SPIs for extensibility (e.g., `SourceOkHttpClientRegistrar`, `GlobalInterceptor`, `RetrofitDegrade`, `ErrorDecoder`), and the configuration is annotation-driven with global defaults overridable per-client. The codebase is well-tested with integration tests using MockWebServer, and documentation is provided in multiple languages. However, the tight coupling to specific versions of Retrofit, OkHttp, and Spring Boot introduces moderate technical drift, as some optional dependencies (e.g., Sentinel 1.8.6, Resilience4j 1.7.1) are older. The library is primarily a framework integration rather than a standalone application, but its design patterns serve as a reference for building Spring Boot auto-configuration with complex SPI chains.
The httr package is an R library for making HTTP requests. It provides functions for the main HTTP verbs (GET, POST, PUT, DELETE, PATCH, HEAD) and a generic VERB function for custom verbs. It includes comprehensive support for HTTP configuration (authentication, headers, cookies, proxies, timeouts, etc.) and OAuth 1.0 and 2.0 authentication. Responses are captured in a response object and can be accessed as raw bytes, text, or parsed into common formats (JSON, XML, HTML, etc.). The package also includes utilities for URL parsing and building, media type parsing, HTTP date handling, retry logic with exponential backoff, and HTTP caching. Note: This package is superseded by httr2 for new development, but is still maintained for CRAN compatibility.