$ show stamparm/maltrail
Maltrail is a production-grade malicious traffic detection system that monitors network traffic in real-time, matching against a dynamic set of threat indicators (trails) and applying behavioral heuristics. It consists of two main components: a high-performance Rust sensor for packet capture and analysis, and a Python server for event intake, web dashboard, and API. The sensor uses libpcap for live capture or offline pcap replay, parses protocols (DNS, HTTP, TLS, QUIC), and performs trail matching and heuristic detection (port scanning, beaconing, DNS tunneling/exhaustion). Events are sent to the server via UDP or stored locally. The server provides a web UI for visualization, search, and management, supports remote logging to SIEMs (syslog, CEF, Logstash), and exposes a REST API for integration. The architecture is modular: the sensor is a multi-threaded Rust binary with lock-free metrics, memory-mapped trail stores, and fuzz-tested protocol parsers. The Python server handles configuration, trail updates from 40+ threat feeds, event logging, and the web interface. The system is designed for scalability with multi-worker capture, bounded memory heuristics, and efficient data structures (LRU caches, interned strings, hash tables). External dependencies include libpcap, SQLite, and various threat feed sources. The codebase demonstrates high craftsmanship with extensive testing (unit, integration, fuzz, parity tests between Rust and Python), CI/CD pipelines, and Docker/systemd deployment support.
The LLM identified 4 files worth highlighting for reuse:
Implements parallel loading of CSV threat indicators into an optimized in-memory store. Uses rayon for parallel parsing, interned strings to reduce duplication, and open-addressing hash tables for fast lookups. The loader handles large datasets (millions of entries) with minimal memory overhead and supports incremental updates. This pattern is reusable for any application that needs to load and query large static datasets efficiently.
Implements port scanning, web scanning, and infection scanning detection using sliding-window accumulators with bounded memory. Tracks connection attempts per source IP within configurable time windows and thresholds. Uses LRU eviction to prevent memory exhaustion under high traffic. The pattern is reusable for any rate-limiting or anomaly detection system that needs to track per-key counters over time.
Provides an LRU (Least Recently Used) dictionary implementation with a configurable maximum size. Automatically evicts least recently accessed items when the limit is reached. Used throughout Maltrail for caching results (e.g., DNS lookups, trail lookups) to bound memory usage. This is a clean, self-contained utility that can be extracted and reused in any Python project needing a bounded cache.
Parses TLS ClientHello messages to extract JA3/JA4 fingerprints, SNI, and cipher suite information. Handles both TLS 1.2 and 1.3, including extensions and compression methods. The parser is fuzz-tested and designed to never panic on malformed input. This component is reusable for any network analysis tool that needs TLS fingerprinting without external dependencies.