Maigret is a username-based OSINT reconnaissance tool that checks across thousands of websites for accounts associated with a given username, extracts profile information via the socid_extractor library, and generates multi-format reports (HTML, PDF, CSV, JSON, Markdown, XMind, Neo4j, etc.). It operates as a CLI tool, an embeddable Python library, a Flask web application with live scan streaming, and a Docker image. The architecture is a monolith with a clean separation between site database management (MaigretDatabase/MaigretSite models), asynchronous checking engine (AsyncioQueueGeneratorExecutor, checkers for aiohttp, curl_cffi, Cloudflare bypass), extraction enrichment, and report generation. External dependencies include aiohttp, socid_extractor, curl_cffi, aiodns, and potentially FlareSolverr and OpenAI-compatible APIs (optional). The tool supports proxy/Tor/I2P routing, self-validation of site configurations, and auto-update of its site database. It is actively maintained, with frequent releases and a large community, and is used by professionals in OSINT, academic research, cryptocurrency investigations, and privacy analysis.
OpenDoor is an advanced CLI security reconnaissance platform for directory discovery, subdomain enumeration, WAF detection, and content fingerprinting. It employs a modular architecture with a plugin-based analysis pipeline (sniffers), network transport abstraction (direct, proxy, VPN), and extensive output formatting (JSON, CSV, SQLite, SARIF). Its core scanning engine, state machine, and plugin systems demonstrate sophisticated engineering, featuring auto-calibration, semantic soft-404 diffing, and a clean exception hierarchy.
Sherlock is a monolithic Python CLI for username enumeration across social networks. For a given username it fans out concurrent HTTP probes to roughly 400 external sites (defined declaratively in sherlock_project/resources/data.json), then classifies each response as Claimed / Available / Unknown / Illegal / WAF using three detection modes: HTTP status code, presence/absence of a site-specific error message in the response body, or redirect behavior (redirects disabled, 2xx/3xx means claimed). Requests run in parallel via a requests-futures session capped at 20 workers; an optional per-site regex short-circuits illegal usernames without sending a request, and hardcoded fingerprint strings detect Cloudflare / PerimeterX / AWS WAF blocks. The CLI accepts one or more usernames, supports {?} wildcard expansion into `_`, `-`, `.` variants, and reports results to the terminal via a QueryNotify observer, with optional .txt, .csv, and .xlsx export. The system is fully data-driven: no per-site code exists. Each manifest entry declares the probe URL, error detection method, error signatures, optional headers/request method/JSON payload, and NSFW flag; a JSON Schema (data.schema.json) enforces conditional consistency (e.g., errorMsg required when errorType includes 'message'). By default the manifest is fetched live from https://data.sherlockproject.xyz, with an ephemeral false-positive exclusion list pulled from a dedicated GitHub branch; --local, --json, and --site flags override this behavior. Entry points are the `sherlock` console script, `python -m sherlock_project`, a Docker image entrypoint, and an Apify Actor (.actor/) that wraps the Docker image and pushes JSON results into an Apify dataset. Engineering strengths include the declarative manifest/schema design, a genuinely useful test suite (schema validation, live-target probe tests, randomized false-positive fuzzing, a Unicode regression test for issue #2730), and CI that runs a tox matrix across Python 3.10-3.14t on three OSes, validates only manifest entries modified by a PR, comments the validation summary back on the PR, and updates the exclusions list daily. Weaknesses: nearly all orchestration and output-format logic lives in a ~1,200-line sherlock.py; notify.py uses a module-global result counter; sites.py has mutable default arguments and a dead username_unclaimed parameter; the --print-found flag is always True (store_true combined with default=True); stem is an unused declared dependency; and the request layer relies on requests-futures, an effectively unmaintained library. The project is actively maintained and extremely popular (93k+ stars), and its health depends on hundreds of external sites remaining stable, hence the extensive removed-sites documentation and dedicated false-positive/false-negative issue templates. Technical drift is moderate rather than severe: the Python ^3.9 floor is now EOL, requests-futures is unmaintained, the WAF fingerprint catalog has not been extended since late 2024, and the PyPI readme still documents removed --tor flags.
OneForAll is a comprehensive subdomain enumeration and reconnaissance tool designed for security researchers and penetration testers. It aggregates subdomains from over 60 passive sources (certificate transparency logs, DNS datasets, search engines, threat intelligence feeds, and web crawlers), performs active brute-forcing via massdns, and enriches results with DNS resolution, HTTP probing, CDN detection, wildcard filtering, and subdomain takeover risk assessment. The tool is built as a modular Python CLI application, with each data source implemented as an independent plugin inheriting from base classes in `common/`. Output is stored in a SQLite database and can be exported to CSV, JSON, XLSX, and other formats. While primarily an application, the repository also includes curated data files (subdomain wordlists, CDN CIDR ranges, cloud service fingerprints) that are valuable for security research.
reNgine is a Django-based automated security reconnaissance and attack-surface management platform. It orchestrates 20+ external security tools (Subfinder, Amass, Naabu, Nuclei, FFUF, GoSpider, Dalfox, WAFW00f, EyeWitness, theHarvester, h8mail) via Celery task chains to perform subdomain discovery, port scanning, HTTP crawling, directory fuzzing, OSINT gathering, technology fingerprinting, WAF detection, and vulnerability scanning. Results are parsed from tool outputs (nmap XML, nuclei JSON, dalfox JSON, ffuf JSON), persisted in PostgreSQL, and exposed through a Django REST Framework API plus a jQuery/Bootstrap frontend with server-side DataTables, dashboards, and WeasyPrint PDF reports. The system supports multi-project tenancy, role-based access control (sys_admin, penetration_tester, auditor), scheduled scans via Celery Beat, and deep HackerOne bug-bounty integration for program import, bookmarking, and vulnerability report submission. It also integrates LLMs (OpenAI and local Ollama) for vulnerability description enrichment and attack-surface suggestions. Architecturally it is a monolithic Django application with four feature apps (dashboard, scanEngine, startScan, targetApp, recon_note) and a Docker deployment composed of web, celery, celery-beat, redis, postgres, nginx, and ollama services. The orchestration core, web/reNgine/tasks.py (~3000 lines), composes Celery groups/chords/chains, executes shell commands against external tools, parses their structured output, and persists scan results. The frontend is server-rendered Django templates with heavy inline JavaScript and a supplementary DRF API layer in web/api/views.py. Although it collects and aggregates large volumes of scan-derived data, this is fundamentally an operational security application, not a data-research project. The codebase is actively patched (recent CVE fixes for command injection and XSS) and feature-rich, but carries notable technical debt: an end-of-life Django 3.2 base, old vendored JS plugins (DataTables 1.10.16, Lightbox from 2018), hardcoded container filesystem paths, a monolithic task module mixing orchestration/parsing/persistence, and dead or commented code. The strongest reusable pieces are the RengineTask Celery base class (standardized task logging, caching, and notifications), the dual-provider LLM abstraction with OpenAI/Ollama backends, and a custom server-side datatable filtering language applying operators (=, >, <, !, &, |) across subdomain, endpoint, and vulnerability views. It is best characterized as effective, pragmatic, security-tool engineering rather than clean, academically elegant architecture.
Seeker is a CLI-based social engineering tool that hosts fake web pages designed to trick users into granting geolocation permissions. When a victim visits the served page, JavaScript collects precise GPS coordinates, device fingerprinting data (OS, CPU cores, RAM, screen resolution, GPU, browser, IP addresses), and public IP information via ipwhois.app. The tool uses PHP's built-in server to serve the templates and handles incoming data through PHP endpoints that write to local log files. Seeker then polls these logs, stores results in CSV/KML formats, and can forward captured data to Telegram bots or Discord webhooks. It supports multiple phishing templates (Google Drive, WhatsApp, Telegram, Zoom, reCAPTCHA, and a 'NearYou' map page) and integrates with tunneling services like ngrok for public exposure. The repository is primarily a penetration testing and educational proof-of-concept, demonstrating the ease of harvesting geolocation data through deceptive web pages. While functional and well-documented, the codebase lacks tests and uses environment-variable-driven configuration to avoid interactive prompts in deployment scenarios.
AssetsHunter is a Python-based information gathering framework designed for asset discovery and reconnaissance. It provides both active and passive modules for tasks such as web service detection (Hawkeye), information leakage checks (InfoRisk), CMS fingerprinting (WhatCMS), and various passive data collection methods including DNS resolution, WHOIS queries, SSL certificate transparency logs, and IP reputation services. The framework uses a modular architecture with command-line argument parsing to select specific reconnaissance modules. While functional, the code exhibits several quality issues: inconsistent error handling, tight coupling between modules and configuration, repetitive code patterns (especially in the passive modules), and limited use of modern Python practices. The framework demonstrates basic reconnaissance capabilities but lacks sophistication in areas like rate limiting, robust error handling, and modular extensibility.