Scrapling is a high-performance Python web scraping library that provides a unified interface for static HTTP requests (via curl_cffi with TLS fingerprint impersonation), dynamic browser automation (Playwright), and stealthy browser sessions with Cloudflare Turnstile/interstitial bypass. It features an adaptive HTML parser (Selector) that can automatically relocate elements after website structure changes using similarity scoring, stored in a SQLite-backed storage system. The library includes a full spider/crawler framework (Spider, CrawlSpider, SitemapSpider, etc.) with concurrent requests, pause/resume via checkpoints, AutoThrottle, proxy rotation, and streaming output. Additionally, it offers a CLI tool for quick extraction, an interactive IPython shell, a Scrapy integration decorator, and an MCP server that exposes 13 scraping tools for AI agents (Claude, Cursor). The codebase is well-structured with clear separation of concerns, extensive type hints, and 92% test coverage. While the library is actively maintained and uses modern Python features, it has some code duplication between sync/async implementations and relies on heavy external dependencies (Playwright, curl_cffi, lxml) that require careful version management.
This repository is a reverse-proxy that translates ChatGPT's proprietary web API into OpenAI-compatible REST endpoints, enabling standard OpenAI SDK clients to generate images, edit images, have chat conversations, and search using ChatGPT's backend. It manages a pool of ChatGPT accounts, handles token refresh, proxy rotation, Cloudflare clearance (FlareSolverr, WARP), and provides a Next.js web dashboard for account management, image generation, settings, and backup. The backend is built with FastAPI and uses curl_cffi for HTTP impersonation, while the frontend is a modern React application with shadcn/ui components. The architecture is modular: protocol adapters in `services/protocol/` convert between OpenAI/Anthropic formats and ChatGPT's internal API, storage backends (JSON, SQLite, PostgreSQL, Git) abstract persistence, and services handle account lifecycle, image processing, and proxy configuration. The system is designed for self-hosted deployment via Docker Compose and supports advanced features like content filtering, chat completion caching, and editable file generation (PPT/PSD). Despite its strong engineering and active maintenance, the codebase is heavily dependent on reverse-engineering of ChatGPT's private API, making it fragile to upstream changes. The large single-file `OpenAIBackendAPI` class and limited test coverage are areas of concern, but the separation of concerns in storage, protocol, and service layers offers good reusability.
CloakBrowser is a commercial stealth browser automation library that wraps Playwright (Python, JavaScript, .NET) with a custom patched Chromium binary. Its core purpose is to evade bot detection by spoofing browser fingerprints, providing human-like interaction (Bezier mouse moves, realistic typing with typos, smooth scrolling), and managing licensed binary downloads with cryptographic verification. The architecture is a multi-language SDK sharing identical logic across Python, TypeScript, and C#, with a common human interaction engine, proxy/GeoIP integration, and license enforcement via denial files and server validation. Key entry points are the `launch()` and `humanize()` APIs, CLI diagnostics (`info`, `login`, `install`), and the `cloakserve` CDP multiplexer. External dependencies include Playwright/Puppeteer, MaxMind GeoLite2 database, Ed25519 signature verification (BouncyCastle), and a license validation server. The codebase demonstrates high engineering discipline with extensive cross-language parity tests and security-conscious binary distribution, but carries drift risks from triple-language duplication and reliance on Playwright internal APIs.
This repository provides a stealth browser automation MCP (Model Context Protocol) server that offers undetectable control over real browser instances (Chrome, Chromium, Edge) via the Chrome DevTools Protocol (CDP) and the `nodriver` library. It exposes 97 tools across 11 sections including navigation, DOM interaction, element extraction (styles, structure, events, animations), network interception and modification (including dynamic Python-powered hooks), CDP command execution, cookie management, tab control, and proxy forwarding with authentication. The architecture is a FastMCP-based server with singleton managers for browser instances, network interception, element cloning, and process cleanup. It supports stdio and HTTP transports, Docker deployment, and optional bearer auth. The system is designed to bypass anti-bot detection (Cloudflare, CreepJS, login walls) and is used for scraping, testing, and AI agent automation. While the codebase is well-structured and the functionality is comprehensive, there is some code duplication across multiple element cloner implementations and technical drift risk due to tight coupling with CDP APIs and the `py2js` transpiler dependency.
UKIKU is a feature-rich Android application for streaming and downloading anime episodes, built primarily in Kotlin with some legacy Java. It scrapes content from animeav1.com (and related sites) using Jsoup and custom JavaScript extraction, bypassing Cloudflare protection via a WebView-based mechanism. The app offers a comprehensive anime tracking experience: browsing a directory, searching, viewing details, watching episodes via ExoPlayer (with Chromecast and Android TV support), downloading (including HLS), managing favorites and watch history, queuing episodes, earning achievements, and syncing user data to Firebase Firestore or Dropbox. It also includes a gamified economy system (loli-coins) and extensive ad integration (AdMob, AppLovin). The architecture follows MVVM with Room database, WorkManager, and coroutines, but suffers from significant technical debt: duplicate UI layouts (material vs. non-material), deprecated libraries (Anko, RxJava, old MaterialDialogs), fragile scraping logic, weak encryption, and hardcoded API keys. Despite these issues, the app is functional and has a large user base (203 stars, 39 forks). It is not a data research project; it is a consumer Android application.