Maigret is a username-based OSINT reconnaissance tool that checks across thousands of websites for accounts associated with a given username, extracts profile information via the socid_extractor library, and generates multi-format reports (HTML, PDF, CSV, JSON, Markdown, XMind, Neo4j, etc.). It operates as a CLI tool, an embeddable Python library, a Flask web application with live scan streaming, and a Docker image. The architecture is a monolith with a clean separation between site database management (MaigretDatabase/MaigretSite models), asynchronous checking engine (AsyncioQueueGeneratorExecutor, checkers for aiohttp, curl_cffi, Cloudflare bypass), extraction enrichment, and report generation. External dependencies include aiohttp, socid_extractor, curl_cffi, aiodns, and potentially FlareSolverr and OpenAI-compatible APIs (optional). The tool supports proxy/Tor/I2P routing, self-validation of site configurations, and auto-update of its site database. It is actively maintained, with frequent releases and a large community, and is used by professionals in OSINT, academic research, cryptocurrency investigations, and privacy analysis.
Linkook is a Python CLI tool for OSINT-driven discovery of linked social media accounts and associated emails across multiple platforms. It takes a single username as input, scans a configurable set of websites (defined in a provider.json file), and recursively extracts linked accounts, usernames, and emails from discovered profiles. The tool supports breach checking via HudsonRock's database or Have I Been Pwned, and can export results to a Neo4j-compatible JSON format for graph visualization. The architecture is modular: a Provider class handles URL templates and regex extraction, a SiteScanner performs HTTP requests and keyword-based existence checks, and a ScannerManager manages concurrent scanning via a thread pool with a queue. The code is well-structured, includes unit tests, and demonstrates solid engineering practices with clean separation of concerns, error handling, and logging.
Favicorn is a command-line OSINT tool that searches for websites by their favicon hashes. It accepts input via favicon URL, file, or domain, computes multiple cryptographic and perceptual hashes (MMH3, MD5, SHA256, etc.), and generates search URLs for 11 security/search platforms (Shodan, ZoomEye, Censys, etc.). Optionally, it fetches and displays preview results from Shodan, ZoomEye, and Netlas using API keys. The architecture features a modular design: the Favicon class handles hash computation and platform-specific URL generation, while specialized Fetcher classes (Shodan, ZoomEye, Netlas) encapsulate API interactions, caching, and response parsing. The CLI orchestrates workflow, parallel execution, and output formatting. Key dependencies include requests, mmh3, dnspython, and platform-specific SDKs.
Sherlock is a monolithic Python CLI for username enumeration across social networks. For a given username it fans out concurrent HTTP probes to roughly 400 external sites (defined declaratively in sherlock_project/resources/data.json), then classifies each response as Claimed / Available / Unknown / Illegal / WAF using three detection modes: HTTP status code, presence/absence of a site-specific error message in the response body, or redirect behavior (redirects disabled, 2xx/3xx means claimed). Requests run in parallel via a requests-futures session capped at 20 workers; an optional per-site regex short-circuits illegal usernames without sending a request, and hardcoded fingerprint strings detect Cloudflare / PerimeterX / AWS WAF blocks. The CLI accepts one or more usernames, supports {?} wildcard expansion into `_`, `-`, `.` variants, and reports results to the terminal via a QueryNotify observer, with optional .txt, .csv, and .xlsx export. The system is fully data-driven: no per-site code exists. Each manifest entry declares the probe URL, error detection method, error signatures, optional headers/request method/JSON payload, and NSFW flag; a JSON Schema (data.schema.json) enforces conditional consistency (e.g., errorMsg required when errorType includes 'message'). By default the manifest is fetched live from https://data.sherlockproject.xyz, with an ephemeral false-positive exclusion list pulled from a dedicated GitHub branch; --local, --json, and --site flags override this behavior. Entry points are the `sherlock` console script, `python -m sherlock_project`, a Docker image entrypoint, and an Apify Actor (.actor/) that wraps the Docker image and pushes JSON results into an Apify dataset. Engineering strengths include the declarative manifest/schema design, a genuinely useful test suite (schema validation, live-target probe tests, randomized false-positive fuzzing, a Unicode regression test for issue #2730), and CI that runs a tox matrix across Python 3.10-3.14t on three OSes, validates only manifest entries modified by a PR, comments the validation summary back on the PR, and updates the exclusions list daily. Weaknesses: nearly all orchestration and output-format logic lives in a ~1,200-line sherlock.py; notify.py uses a module-global result counter; sites.py has mutable default arguments and a dead username_unclaimed parameter; the --print-found flag is always True (store_true combined with default=True); stem is an unused declared dependency; and the request layer relies on requests-futures, an effectively unmaintained library. The project is actively maintained and extremely popular (93k+ stars), and its health depends on hundreds of external sites remaining stable, hence the extensive removed-sites documentation and dedicated false-positive/false-negative issue templates. Technical drift is moderate rather than severe: the Python ^3.9 floor is now EOL, requests-futures is unmaintained, the WAF fingerprint catalog has not been extended since late 2024, and the PyPI readme still documents removed --tor flags.
OneForAll is a comprehensive subdomain enumeration and reconnaissance tool designed for security researchers and penetration testers. It aggregates subdomains from over 60 passive sources (certificate transparency logs, DNS datasets, search engines, threat intelligence feeds, and web crawlers), performs active brute-forcing via massdns, and enriches results with DNS resolution, HTTP probing, CDN detection, wildcard filtering, and subdomain takeover risk assessment. The tool is built as a modular Python CLI application, with each data source implemented as an independent plugin inheriting from base classes in `common/`. Output is stored in a SQLite database and can be exported to CSV, JSON, XLSX, and other formats. While primarily an application, the repository also includes curated data files (subdomain wordlists, CDN CIDR ranges, cloud service fingerprints) that are valuable for security research.
reNgine is a Django-based automated security reconnaissance and attack-surface management platform. It orchestrates 20+ external security tools (Subfinder, Amass, Naabu, Nuclei, FFUF, GoSpider, Dalfox, WAFW00f, EyeWitness, theHarvester, h8mail) via Celery task chains to perform subdomain discovery, port scanning, HTTP crawling, directory fuzzing, OSINT gathering, technology fingerprinting, WAF detection, and vulnerability scanning. Results are parsed from tool outputs (nmap XML, nuclei JSON, dalfox JSON, ffuf JSON), persisted in PostgreSQL, and exposed through a Django REST Framework API plus a jQuery/Bootstrap frontend with server-side DataTables, dashboards, and WeasyPrint PDF reports. The system supports multi-project tenancy, role-based access control (sys_admin, penetration_tester, auditor), scheduled scans via Celery Beat, and deep HackerOne bug-bounty integration for program import, bookmarking, and vulnerability report submission. It also integrates LLMs (OpenAI and local Ollama) for vulnerability description enrichment and attack-surface suggestions. Architecturally it is a monolithic Django application with four feature apps (dashboard, scanEngine, startScan, targetApp, recon_note) and a Docker deployment composed of web, celery, celery-beat, redis, postgres, nginx, and ollama services. The orchestration core, web/reNgine/tasks.py (~3000 lines), composes Celery groups/chords/chains, executes shell commands against external tools, parses their structured output, and persists scan results. The frontend is server-rendered Django templates with heavy inline JavaScript and a supplementary DRF API layer in web/api/views.py. Although it collects and aggregates large volumes of scan-derived data, this is fundamentally an operational security application, not a data-research project. The codebase is actively patched (recent CVE fixes for command injection and XSS) and feature-rich, but carries notable technical debt: an end-of-life Django 3.2 base, old vendored JS plugins (DataTables 1.10.16, Lightbox from 2018), hardcoded container filesystem paths, a monolithic task module mixing orchestration/parsing/persistence, and dead or commented code. The strongest reusable pieces are the RengineTask Celery base class (standardized task logging, caching, and notifications), the dual-provider LLM abstraction with OpenAI/Ollama backends, and a custom server-side datatable filtering language applying operators (=, >, <, !, &, |) across subdomain, endpoint, and vulnerability views. It is best characterized as effective, pragmatic, security-tool engineering rather than clean, academically elegant architecture.
AssetsHunter is a Python-based information gathering framework designed for asset discovery and reconnaissance. It provides both active and passive modules for tasks such as web service detection (Hawkeye), information leakage checks (InfoRisk), CMS fingerprinting (WhatCMS), and various passive data collection methods including DNS resolution, WHOIS queries, SSL certificate transparency logs, and IP reputation services. The framework uses a modular architecture with command-line argument parsing to select specific reconnaissance modules. While functional, the code exhibits several quality issues: inconsistent error handling, tight coupling between modules and configuration, repetitive code patterns (especially in the passive modules), and limited use of modern Python practices. The framework demonstrates basic reconnaissance capabilities but lacks sophistication in areas like rate limiting, robust error handling, and modular extensibility.
This repository contains only documentation and configuration files (README, CONTRIBUTING, issue templates, .gitignore) with no actual source code present. The README describes SpiderSuite as a cross-platform web crawler for security professionals, but the codebase itself is empty in this snapshot. No functional analysis is possible. The repository appears to be a stub or the source code is hosted elsewhere. Given the absence of any executable or library code, this repository provides no reusable software assets.