garak is a CLI tool and Python library for automated red-teaming and vulnerability scanning of large language models (LLMs). It systematically probes models with adversarial inputs (jailbreaks, prompt injections, encoding attacks, etc.) and uses detectors to classify model responses for safety failures. It supports multiple model backends (OpenAI, Anthropic, HuggingFace, etc.) and generates structured reports with severity scoring. The system follows a modular plugin architecture with base classes for probes, detectors, generators, harnesses, evaluators, and buffs. Plugins are dynamically loaded and configured via a hierarchical config system. The default harness runs probes with their associated detectors, while a cross-product harness allows all probes against all detectors. Results are logged as JSONL and can be visualized via a React-based web UI (garak-report) that displays DEFCON-level severity, calibration Z-scores, and technique-intent matrices. While primarily an application, garak incorporates significant research elements: it implements state-of-the-art attack techniques (GCG, AutoDAN, BEAST, TAP) and includes curated datasets of adversarial prompts and calibration data. The codebase serves both as a practical security tool and as a research platform for LLM vulnerability assessment.
ClawScan is a Go-based CLI tool for composable security scanning of AI agent skills and plugins. It orchestrates multiple external scanners (e.g., Snyk, VirusTotal, Socket, AIG, Cisco, SkillSpector) against skill directories, collects raw evidence, and optionally feeds results to an LLM judge for evaluation. The tool supports profile-based configuration (built-in profiles like `clawhub` and `openclaw-install-policy`), Docker sandboxing, declarative gate rules for automated policy enforcement, and benchmark execution against datasets like SkillTrustBench and clawhub-security-signals. It is distributed via npm (`@openclaw/clawscan`) and GitHub Releases, with a rich CI/CD pipeline including automated runtime image builds, dependency updates, and a Hugging Face leaderboard for security signals research. While primarily an application, it has strong data/research elements through its benchmark infrastructure and leaderboard submission validation.
Nova Proximity is a modular Python security scanner designed specifically for MCP (Model Context Protocol) servers and Agent Skills. The application provides comprehensive discovery and analysis capabilities, including tool/prompt/resource enumeration for MCP servers and structural/permission analysis for Agent Skills. Its architecture separates concerns into distinct libraries: mcp_scanner_lib for MCP protocol handling, skill_scanner_lib for Agent Skills analysis, and nova_evaluator_lib for NOVA-powered security evaluation. The tool supports multiple transport mechanisms (stdio, SSE, Streamable HTTP) with automatic fallback, provides detailed function signature analysis, and generates actionable security reports with pattern-specific remediation guidance.
Agentic Security is an open-source LLM vulnerability scanner that probes AI endpoints for security weaknesses such as jailbreaks, prompt injection, data leakage, and sandbox escapes. It provides both a FastAPI web server (port 8718) with a Vue.js frontend and a Fire-based CLI (server, ci, init, ls, scan). The core scan engine sends adversarial prompts (text, image, audio) to target LLMs via configurable HTTP specs, classifies responses using a multi-layered refusal detection system (phrase-based, OneClassSVM, PII detector, LLM-as-judge, hybrid voting), and generates detailed reports with failure rates and cost metrics. The architecture is modular: LLM provider abstraction (OpenAI, Anthropic, LiteLLM), resilience primitives (rate limiter, circuit breaker, concurrent executor), plugin-based refusal classifiers, and a dataset registry supporting HuggingFace, CSV, Google Sheets, and dynamic transformations (stenography, encoding). It also integrates external red-teaming tools (Garak, Inspect AI) and includes reinforcement learning and Bayesian optimization for adaptive prompt selection. While primarily an application, it contains significant data/research components (ML models, datasets, refusal classification research) that support its scanning mission.
The repository is a comprehensive platform for AI red teaming education, hosting a series of Capture the Flag (CTF) challenges that target LLM vulnerabilities such as prompt injection, metaprompt extraction, and safety filter bypass. It is built as a modified version of Microsoft's Chat Copilot, integrating a multi-service architecture: a .NET Core chat backend with Semantic Kernel, a React frontend, a load balancer (Go), a scoring service (Flask/Celery), a picture submission service, and a challenge home page. The system is designed to be deployed via Docker Compose or Kubernetes, with each challenge running its own configured chat instance. While primarily an application, the repository also includes Jupyter notebooks using PyRIT for automated red teaming, blending application and research elements. The architecture is modular, with clear separation between services. The chat copilot backend provides real-time chat via SignalR, supports OpenAPI plugins, and integrates with Azure services (CosmosDB, Redis, Blob Storage, Content Safety). The scoring system supports both automatic (regex/LLM) and manual review, with CTFd flag submission. The platform is highly configurable through challenge JSON files, allowing different metaprompts, scorers, RAG inputs, and security settings. This makes it a flexible testbed for red teaming exercises. The repository demonstrates strong engineering practices with reusable components such as a generic repository pattern, authentication handlers, telemetry services, and a load balancer. However, it is tightly coupled to Azure and Semantic Kernel, and some features are intentionally insecure for educational purposes. The codebase is actively maintained and well-documented, making it suitable for both learning and running red teaming events.
LLMMap is a CLI-based automated prompt injection testing framework for LLM-integrated applications, modeled after sqlmap. It systematically discovers injection points (query parameters, headers, body, cookies, path), generates goal-aware adversarial prompts using a dual-LLM architecture (Generator + Judge), and evaluates responses through pattern matching, semantic similarity, and out-of-band callbacks. The tool supports 227 injection techniques across 18 attack families, multiple LLM providers (Ollama, OpenAI, Anthropic, Google), and provides reliability confirmation via Wilson confidence intervals. It includes comprehensive reporting, data-flow tracing, and Burp Suite integration. The codebase is well-structured with clean abstractions, extensive tests, and reusable components, though some modules are reserved for future development and external dependencies like interactsh-client introduce fragility.