Agentic Security is an open-source LLM vulnerability scanner that probes AI endpoints for security weaknesses such as jailbreaks, prompt injection, data leakage, and sandbox escapes. It provides both a FastAPI web server (port 8718) with a Vue.js frontend and a Fire-based CLI (server, ci, init, ls, scan). The core scan engine sends adversarial prompts (text, image, audio) to target LLMs via configurable HTTP specs, classifies responses using a multi-layered refusal detection system (phrase-based, OneClassSVM, PII detector, LLM-as-judge, hybrid voting), and generates detailed reports with failure rates and cost metrics. The architecture is modular: LLM provider abstraction (OpenAI, Anthropic, LiteLLM), resilience primitives (rate limiter, circuit breaker, concurrent executor), plugin-based refusal classifiers, and a dataset registry supporting HuggingFace, CSV, Google Sheets, and dynamic transformations (stenography, encoding). It also integrates external red-teaming tools (Garak, Inspect AI) and includes reinforcement learning and Bayesian optimization for adaptive prompt selection. While primarily an application, it contains significant data/research components (ML models, datasets, refusal classification research) that support its scanning mission.
The repository is a comprehensive platform for AI red teaming education, hosting a series of Capture the Flag (CTF) challenges that target LLM vulnerabilities such as prompt injection, metaprompt extraction, and safety filter bypass. It is built as a modified version of Microsoft's Chat Copilot, integrating a multi-service architecture: a .NET Core chat backend with Semantic Kernel, a React frontend, a load balancer (Go), a scoring service (Flask/Celery), a picture submission service, and a challenge home page. The system is designed to be deployed via Docker Compose or Kubernetes, with each challenge running its own configured chat instance. While primarily an application, the repository also includes Jupyter notebooks using PyRIT for automated red teaming, blending application and research elements. The architecture is modular, with clear separation between services. The chat copilot backend provides real-time chat via SignalR, supports OpenAPI plugins, and integrates with Azure services (CosmosDB, Redis, Blob Storage, Content Safety). The scoring system supports both automatic (regex/LLM) and manual review, with CTFd flag submission. The platform is highly configurable through challenge JSON files, allowing different metaprompts, scorers, RAG inputs, and security settings. This makes it a flexible testbed for red teaming exercises. The repository demonstrates strong engineering practices with reusable components such as a generic repository pattern, authentication handlers, telemetry services, and a load balancer. However, it is tightly coupled to Azure and Semantic Kernel, and some features are intentionally insecure for educational purposes. The codebase is actively maintained and well-documented, making it suitable for both learning and running red teaming events.