$ show g0ldyy/comet
Comet is a production-grade Stremio add-on (and Kodi plugin) that functions as a high-performance torrent and debrid streaming proxy. It aggregates torrent metadata from 15+ indexers (Torrentio, Jackett, Prowlarr, Nyaa, DMM, etc.), caches results intelligently, and optionally shares discovered metadata across a custom P2P network (CometNet). The architecture is a FastAPI web server with multiple background workers for proactive scraping, metadata enrichment (TMDB, Kitsu, Cinemeta), debrid account syncing, and automated DMM hashlist ingestion. It provides a built-in streaming proxy with per-IP bandwidth controls, an admin dashboard for monitoring and control, and integrates with Prowlarr/Radarr/Sonarr via Torznab. The codebase is well-structured with comprehensive test coverage, Pydantic validation throughout, async-first patterns, and a clean scraper abstraction layer. It is actively maintained with frequent commits and a mature feature set, though the CometNet P2P component remains experimental and some external scrapers may be fragile due to Cloudflare protections.
The LLM identified 4 files worth highlighting for reuse:
Implements a database-backed distributed lock using SQLite or PostgreSQL, with automatic renewal, timeout enforcement, and a context manager (`run`) that ensures exclusive access for critical operations like background scraper cycles and DMM ingester runs. It is self-contained, uses only standard library and the core database module, and can be extracted into another project with minimal changes.
Provides an `AsyncClientWrapper` and `NetworkManager` that encapsulate aiohttp and curl_cffi backends with automatic fallback, proxy management (via `ProxyEthos`), 429 retry logic, and configurable timeouts. It is designed as a shared singleton but the pattern of a configurable async HTTP client with retry and proxy rotation is reusable in any Python web scraping project.
Defines `BaseScraper` abstract class and helper functions (`deduplicate_torrents`, `gather_with_error_logging`) that all concrete scrapers inherit. It enforces a uniform `scrape(request: ScrapeRequest) -> AsyncIterator[TorrentResult]` interface, handles deduplication, logging, and error logging. This pattern is reusable as a template for building any set of data source scrapers.
Contains the `MigrationContext` class and a list of migration functions that manage schema evolution across SQLite and PostgreSQL. It supports column add/drop/rename, index creation, data backfill, and managed table replacement. The pattern is self-contained and can be extracted as a lightweight schema migration framework for small to medium projects.