$ show Hellsender01/LLMMap
LLMMap is a CLI-based automated prompt injection testing framework for LLM-integrated applications, modeled after sqlmap. It systematically discovers injection points (query parameters, headers, body, cookies, path), generates goal-aware adversarial prompts using a dual-LLM architecture (Generator + Judge), and evaluates responses through pattern matching, semantic similarity, and out-of-band callbacks. The tool supports 227 injection techniques across 18 attack families, multiple LLM providers (Ollama, OpenAI, Anthropic, Google), and provides reliability confirmation via Wilson confidence intervals. It includes comprehensive reporting, data-flow tracing, and Burp Suite integration. The codebase is well-structured with clean abstractions, extensive tests, and reusable components, though some modules are reserved for future development and external dependencies like interactsh-client introduce fragility.
The LLM identified 4 files worth highlighting for reuse:
Unified LLM client with pluggable adapters for Ollama, OpenAI, Anthropic, and Google. Encapsulates authentication, request formatting, and response parsing. Reusable in any project requiring multi-provider LLM access without vendor lock-in.
Loads YAML prompt packs with schema validation and caching. Supports versioning, intensity-based selection, and metadata extraction. Reusable for any project that needs to manage and serve prompt templates from external files.
Thread-safe logging utility with queue handler, output buffering, ANSI color support, and configurable verbosity. Designed for CLI tools with concurrent operations. Reusable as a drop-in logger for any Python application needing structured, colorized output.
Implements Wilson score confidence interval calculation for binary outcomes, used to confirm findings with statistical rigor. Includes configurable thresholds and retry logic. Reusable in any application requiring statistical significance testing, such as A/B testing or anomaly detection.