Maltrail is a production-grade malicious traffic detection system that monitors network traffic in real-time, matching against a dynamic set of threat indicators (trails) and applying behavioral heuristics. It consists of two main components: a high-performance Rust sensor for packet capture and analysis, and a Python server for event intake, web dashboard, and API. The sensor uses libpcap for live capture or offline pcap replay, parses protocols (DNS, HTTP, TLS, QUIC), and performs trail matching and heuristic detection (port scanning, beaconing, DNS tunneling/exhaustion). Events are sent to the server via UDP or stored locally. The server provides a web UI for visualization, search, and management, supports remote logging to SIEMs (syslog, CEF, Logstash), and exposes a REST API for integration. The architecture is modular: the sensor is a multi-threaded Rust binary with lock-free metrics, memory-mapped trail stores, and fuzz-tested protocol parsers. The Python server handles configuration, trail updates from 40+ threat feeds, event logging, and the web interface. The system is designed for scalability with multi-worker capture, bounded memory heuristics, and efficient data structures (LRU caches, interned strings, hash tables). External dependencies include libpcap, SQLite, and various threat feed sources. The codebase demonstrates high craftsmanship with extensive testing (unit, integration, fuzz, parity tests between Rust and Python), CI/CD pipelines, and Docker/systemd deployment support.
IntelOwl is a production-grade, open-source threat intelligence platform that automates the analysis of observables (IPs, domains, URLs, hashes) and files (PE, PDF, APK, etc.) through a pluggable plugin system. The backend is built with Django REST Framework, Celery for asynchronous task execution, and Django Channels for real-time WebSocket updates. The frontend is a React single-page application using react-router, react-flow for pipeline visualizations, and Zustand for state management. The system supports multi-tenancy, organizations, role-based access, and integrates with dozens of external threat intelligence APIs and open-source analysis tools via containerized microservices. The architecture is centered around a plugin framework with five plugin types: analyzers (file and observable), connectors (push results to external systems), visualizers (render reports), pivots (create child jobs from analysis results), and ingestors (pull threat feeds). Playbooks bundle these plugins into repeatable analysis workflows. A data model manager normalizes analyzer outputs into structured evaluations with reliability scores, and a user event system allows manual verdict overrides with decay logic. The platform also includes an AI chatbot assistant powered by a local Ollama LLM and LangGraph, enabling natural-language queries and one-click analysis launches. Key operational features include a comprehensive REST API, token-based authentication with optional OAuth/LDAP/RADIUS, Elasticsearch for BI reporting, and Docker-based deployment with separate containers for specialized analyzers. The codebase is actively maintained with extensive unit and integration tests, CI/CD pipelines, and a large community (4695 stars). While the migration system is somewhat repetitive, the overall architecture is clean, modular, and demonstrates strong software engineering practices.
OpenCTI is an open-source cyber threat intelligence platform that ingests, manages, and analyzes threat data using the STIX 2.0/2.1 standard. The system is a monorepo containing a Node.js/TypeScript GraphQL backend (opencti-graphql), a React/Relay frontend (opencti-front), a Python client library (client-python), and a Python worker service (opencti-worker). The backend uses Elasticsearch/OpenSearch as its primary data store, Redis for caching and pub/sub, RabbitMQ for connector queues, and PostgreSQL for migrations. The frontend provides full CRUD UIs, dashboards, workspaces, playbook automation, and workflow management. Key entry points include the GraphQL API (resolvers per entity type), the frontend React app with Relay data fetching, and the Python client for programmatic access. Data flows involve ingestion via CSV/JSON/TAXII/RSS feeds, manual creation, connector enrichment, and inference rules that automatically derive relationships. External dependencies include Elasticsearch, Redis, RabbitMQ, MinIO/S3 for file storage, and optional AI services. The platform supports enterprise features (playbooks, PIR, XTM Hub) gated by license checks.
The LOLBAS project is a curated dataset of Windows binaries, scripts, and libraries that can be abused for malicious purposes (Living Off The Land Binaries and Scripts). Each entry is documented in YAML files with a standardized schema, including technical details, use cases, MITRE ATT&CK mappings, detection artifacts, and external references. The repository serves as a threat intelligence resource for offensive security research and defensive security operations, enabling threat hunting and detection rule development. Data is organized by file type (binaries, libraries, scripts) and maintained through community contributions with automated validation via Pydantic schemas.
T-Pot Community Edition (tpotce) is an all-in-one honeypot platform that orchestrates 20+ honeypots (e.g., cowrie, dionaea, conpot, heralding) alongside network security monitoring tools (p0f, Suricata, fatt) and the Elastic Stack (Elasticsearch, Logstash, Kibana) for centralized log storage, analysis, and real-time attack visualization. The system is deployed via Docker Compose with an Ansible-based installer supporting multiple Linux distributions. It offers several deployment flavors (standard, sensor, mini, LLM, tarpit, mobile) and supports a distributed Hive/Sensor architecture where remote sensors forward encrypted logs to a central Hive over HTTPS. Architecturally, each honeypot runs in an isolated container with read-only filesystems, non-root user, and dedicated networks. The ELK stack processes all telemetry: Logstash ingests JSON logs from persistent volumes, Elasticsearch indexes them, and Kibana provides dashboards and an attack map. A custom p0f (passive OS fingerprinting) fork is bundled and compiled from C source, demonstrating deep network-level analysis. The update and backup system (`update.sh`) is a robust, self-contained script that handles version pinning, configuration backups, Elasticsearch state exports, and image rollouts. The codebase includes a reusable test harness (`docker/_tests/lib/common.sh`) for validating individual honeypot containers with protocol-specific probes.
Yeti is a threat intelligence platform that ingests, enriches, correlates, and stores cyber threat observables, entities, indicators (YARA, Sigma, Suricata, regex, forensic artifacts), and DFIQ investigative questions. It provides a FastAPI-based REST API for CRUD operations, graph traversal, semantic search (via ChromaDB), and RBAC. The system uses ArangoDB as its primary graph/document database, Redis for message brokering, and Celery for background task scheduling. It includes a plugin architecture for feeds (30+ public sources like MITRE ATT&CK, CISA KEV, MISP, AbuseIPDB) and analytics (Shodan, Censys, VirusTotal, etc.). The codebase is well-structured with clean abstractions (ArangoYetiConnector, event system, polymorphic schemas, migration framework) but contains deprecated plugins and some configuration drift. It is primarily an application, not a data research project.
This repository is a curated collection of threat intelligence indicators of compromise (IOCs) published by ESET Research. It contains structured data for hundreds of malware families and APT campaigns, including file hashes (MD5, SHA1, SHA256), network indicators (domains, IPs, URLs), host-based artifacts (registry keys, file paths, mutexes), and detection rules (YARA, Sigma, Snort). The data is organized per-threat in consistent directory structures with README.adoc documentation, MISP event JSON exports, and plain hash lists. The repository serves as a reference for threat hunters, SOC analysts, and security researchers to detect and investigate known threats. While the data is static and time-sensitive, the detection rules and structured IOC formats are directly reusable in security tools and platforms.
Favicorn is a command-line OSINT tool that searches for websites by their favicon hashes. It accepts input via favicon URL, file, or domain, computes multiple cryptographic and perceptual hashes (MMH3, MD5, SHA256, etc.), and generates search URLs for 11 security/search platforms (Shodan, ZoomEye, Censys, etc.). Optionally, it fetches and displays preview results from Shodan, ZoomEye, and Netlas using API keys. The architecture features a modular design: the Favicon class handles hash computation and platform-specific URL generation, while specialized Fetcher classes (Shodan, ZoomEye, Netlas) encapsulate API interactions, caching, and response parsing. The CLI orchestrates workflow, parallel execution, and output formatting. Key dependencies include requests, mmh3, dnspython, and platform-specific SDKs.
Bumblebee is a production-grade supply-chain inventory collector for macOS and Linux developer endpoints. It scans the filesystem for package metadata from lockfiles (npm, pnpm, yarn, bun, gomod, rubygems, composer), install receipts (Homebrew, Python dist-info, Go sum), extension manifests (VS Code, browser extensions), and MCP configuration files. The system outputs NDJSON records of installed packages and can optionally match them against operator-supplied threat intelligence catalogs to emit findings for known malicious packages. It is a single Go binary with zero runtime dependencies, designed for offline, read-only operation. Architecturally, it follows a modular scanner pattern with a generic filesystem walker, per-ecosystem scanners, an exposure catalog loader, and a pluggable output sink (stdout, file, or HTTP with bearer/HMAC auth and gzip). The repository also includes a threat_intel directory of curated supply-chain attack data and a tool to convert OSV records into its catalog format. Although primarily an application, it has a strong data component through its structured threat intelligence catalogs, making it a hybrid that leans towards a security scanning tool.
openSquat is a CLI tool for detecting domain squatting threats including phishing, typosquatting, IDN homograph attacks, doppelgänger domains, and bitsquatting. It operates in three modes: Community (free NRD feed), Premium Feed (paid NRD feed with local Levenshtein detection), and Premium API (server-side lookalike API queries). The tool processes keywords against domain feeds using optimized Levenshtein distance with early exit, integrates with VirusTotal, Quad9 DNS, Certificate Transparency, and phishing databases, and outputs results in TXT, JSON, or CSV formats. Architecture features modular components for feed management, DNS validation, squatting detection, API client, and output handling, with parallel processing for performance.
GreedyBear is a threat intelligence platform that consumes honeypot logs from T-Pot clusters (Cowrie, Heralding, Tanner, etc.), extracts Indicators of Compromise (IOCs), enriches them with external threat feeds (ThreatFox, AbuseIPDB, FireHol, Spamhaus, MalwareBazaar), and applies machine learning scoring (Random Forest) to predict recurrence probability and expected interactions. The system exposes public and authenticated APIs in multiple formats (JSON, NDJSON, CSV, TXT, STIX 2.1) for threat researchers and supports event ingestion from external sensors. The backend is built with Django/DRF, PostgreSQL, and Elasticsearch, orchestrated via Django-Q cronjobs for scheduled extraction, enrichment, and cleanup. A React SPA frontend provides a customizable dashboard with attack maps, trending attackers, and feed browsing. The architecture uses repository, strategy, and factory patterns, with a clean separation between extraction pipelines, scoring, and API layers. While primarily an application, it has strong data engineering components—ETL pipelines, ML scoring, clustering (MinHash LSH for command sequences), and data enrichment—making it a hybrid that leans application.
OSINT Brazuca Regex is a CLI tool and curated dataset of regular expressions for detecting Brazilian and global PII/OSINT-relevant patterns. The scanner (`osint_regex_scan.py`) applies the dataset to files, directories, or stdin, supporting multi-threading, text extraction from documents (PDF, DOCX, XLSX, RTF, EPUB) via stdlib or optional `pypdf`/`pdftotext`, and noise filtering (blob masking, line length limits). Output formats include terminal, CSV, JSON, or raw values. The dataset is split into two JSON files (Brazil-specific and general), merged at load time, and includes patterns for CPF, CNPJ, credit cards, crypto wallets, API credentials, and more. The repository also includes a build script (`build.py`) for managing the split/merged dataset. Core functionality requires only Python 3.12+ stdlib, with optional dependencies for extended PDF support and encoding detection.
IPQuality is a comprehensive bash-based CLI tool for evaluating the quality and risk profile of an IP address. It performs geolocation lookups (via Maxmind and IPinfo), IP type classification (ISP, hosting, business, etc.), risk scoring from multiple threat intelligence providers (AbuseIPDB, IP2Location, IPQS, Scamalytics, etc.), streaming media unlock detection (TikTok, Disney+, Netflix, YouTube, Amazon Prime Video, Reddit, ChatGPT), email service connectivity tests (Gmail, Outlook, Yahoo, etc.), and DNS blacklist checks against 400+ databases. The script is designed for VPS owners and network administrators to quickly assess whether an IP is clean, blocked, or associated with proxies/VPNs. It supports dual-stack IPv4/IPv6, multiple operating systems, Docker, and various output formats (ANSI, JSON, plain text). The architecture is a single monolithic bash script (~78000 tokens) with modular functions, progress bars, and extensive error handling. While not a reusable library, it demonstrates advanced bash scripting patterns for network diagnostics.
MSTICpy is a Python library for cybersecurity threat hunting and investigation, primarily designed for use in Jupyter Notebooks. It provides a unified interface to query multiple data sources (Azure Sentinel, Splunk, Kusto, MDE, etc.), enrich data with threat intelligence and geolocation, perform anomaly detection and process tree analysis, and visualize results with interactive timelines, maps, and graphs. The architecture follows a modular, plugin-based design: data drivers abstract away backend-specific protocols, threat intelligence providers are registered via a consistent base class, and a pivot framework dynamically attaches investigation functions to entity types. The library also includes configuration editors, Azure authentication helpers, and notebook widgets to streamline analyst workflows. While the core purpose is security analysis, the repository also contains substantial research-oriented components, such as anomalous sequence detection (Markov models on command sessions), time-series anomaly detection, and event clustering. These are packaged alongside the production-grade query and enrichment infrastructure, making the repo a hybrid of an application library and a data analysis toolkit. The codebase is well-tested with extensive unit and integration tests, though many require mocked or optional external services. Key entry points include the `QueryProvider` class for data acquisition, `TILookup` for threat intelligence, `Pivot` for entity-driven investigations, and the `mp` pandas accessor for inline transformations. External dependencies span Azure SDK, Bokeh, ipywidgets, and multiple third-party APIs, which introduces some maintenance overhead. Overall, MSTICpy is a mature, actively maintained tool that demonstrates strong engineering practices with reusable patterns, but some modules show signs of version drift and compatibility workarounds.
This repository is a comprehensive library of machine-readable taxonomies (machine tags) for threat intelligence and information sharing, primarily designed for the MISP platform. It contains over 200 taxonomy definitions in a standardized JSON format, each defining a namespace with predicates and values for tagging events, attributes, and indicators. The taxonomies cover a wide range of domains including cyber incident classification, malware analysis, information handling policies (TLP, PAP, IEP), regulatory compliance (GDPR, NIS2, EU AI Act), attack modeling (kill chain, Diamond Model), and specialized areas like disinformation, AI safety, and drone classification. While the core content is data, the repository also includes a set of reusable Python tools for validation, generation, and conversion of taxonomies, as well as CI/CD pipelines and a JSON schema. The hybrid nature is evident: it is primarily a data library but with significant application-like tooling for maintaining and extending the taxonomy collection.
SpyGuard is a network traffic analysis tool designed for detecting signs of compromise, particularly stalkerware and spyware, by monitoring WiFi traffic from connected devices. It consists of a frontend Vue.js 3 application for user interaction and capture initiation, and a backend Flask API for managing configuration, threat intelligence, and analysis. The core analysis engine processes PCAP files using Suricata, checks traffic against IOC databases and whitelists, performs heuristic and active checks (passive DNS, WHOIS, SSL validation), and generates JSON/PDF reports. External watchers periodically update threat intelligence feeds from sources like Cisco Umbrella and IP-to-ASN mappings. The architecture separates concerns: frontend handles capture UI and device management, backend provides authenticated APIs for system administration, and the analysis engine performs the detection logic. Key entry points include server/frontend/main.py (frontend service), server/backend/main.py (backend API), server/backend/watchers.py (threat intelligence updater), and analysis/analysis.py (analysis trigger).
The AIL (Advanced Information Leak) framework is a threat intelligence platform designed for collecting, processing, analyzing, and correlating digital artifacts from various sources including web crawlers, darknet content, forums, chats, and file submissions. It features a modular architecture with a Flask-based web interface, REST API, and background processing workers that communicate via Redis queues. The system extracts entities and relationships from raw data, stores them in Kvrocks (with Redis for caching), provides correlation analysis, and exports processed information to threat intelligence platforms like MISP and TheHive. Key components include extensible processing modules, a flexible object modeling system, and robust queue management for distributed processing.
This repository, branded as "The Malware Bible," is a curated educational knowledge base for cybersecurity, malware analysis, reverse engineering, and exploit development. It is not a conventional software application or an operational data pipeline; under the available classifications it is closest to data/research because its core assets are reference documentation, course-style Markdown modules, and research-oriented binaries rather than runnable application code. The content is organized into four learning paths: The Beginning (C, Python, x86 assembly, shellcode, reverse engineering basics), The Journey (PE/ELF structures, Android/iOS package formats), The Rituals (obfuscation, evasion, and writing malware), and The Scriptures (tool cheatsheets for radare2, IDA Pro, WinDbg, YARA, WinAPI, and evasion tactics). In practice, the repository serves as a self-study curriculum and analyst reference, not as a deployable system. It includes a large Windows DLL export catalog in Markdown, example ELF binaries demonstrating heap overflow, use-after-free, ROP chains and format string bugs, and numerous screenshots/diagrams. There is no CI/CD, dependency management, or automated testing; several files are marked "Coming soon" or "WORK IN PROGRESS," and external links or sponsor images may rot over time. The Windows API and DLL reference content is broad but may drift as Windows evolves. The last commit is recent, indicating active maintenance, and the repository benefits from a structured table of contents and consistent formatting across chapters.
This repository is a curated 'awesome list' of public security-related APIs, not a software application or data pipeline. The primary artifact is README.md, a hand-maintained table of roughly 100 APIs categorized into Online services, Tools, SIEM platforms, and Various. Each entry records the API name, description, authentication type, HTTPS support, documentation link, and licensing model. The only executable logic is a GitHub Actions workflow that runs awesome_bot weekly to detect broken links in the README. A supplementary document, apidocs/haveIBeenPwned.md, documents unofficial Have I Been Pwned domain-search endpoints discovered in 2018. Despite being classified as an application for tooling purposes, there is no application code, library, or service in this repository. It functions as a knowledge base for security engineers and threat intelligence practitioners. The CI workflow is the only automated component and is a standard awesome-list link checker. The value is in the curation and categorization of APIs, not in reusable software components. The list is actively maintained, but some entries are aging and may reflect outdated API details or vendor changes.
This repository is a curated list (awesome list) of open-source intelligence (OSINT) tools and resources. It is not a software application or a data research pipeline; it is a static markdown document that categorizes and links to hundreds of OSINT-related websites, tools, and services. The list covers a wide range of categories including search engines, social media tools, domain and IP research, people investigations, email search, phone number research, threat intelligence, and more. It is maintained by the community through pull requests and follows the awesome-list format. As a curated list, there is no executable code, no data ingestion, no transformation logic, and no analytical models. The value lies in the curation and organization of resources. The repository is primarily a reference for OSINT practitioners, researchers, and security professionals. While not a traditional software project, the list is well-structured and actively maintained. It can be easily cloned and used offline. However, from a software engineering perspective, there are no reusable components or architectural patterns to evaluate.