$ show NVIDIA/garak
garak is a CLI tool and Python library for automated red-teaming and vulnerability scanning of large language models (LLMs). It systematically probes models with adversarial inputs (jailbreaks, prompt injections, encoding attacks, etc.) and uses detectors to classify model responses for safety failures. It supports multiple model backends (OpenAI, Anthropic, HuggingFace, etc.) and generates structured reports with severity scoring. The system follows a modular plugin architecture with base classes for probes, detectors, generators, harnesses, evaluators, and buffs. Plugins are dynamically loaded and configured via a hierarchical config system. The default harness runs probes with their associated detectors, while a cross-product harness allows all probes against all detectors. Results are logged as JSONL and can be visualized via a React-based web UI (garak-report) that displays DEFCON-level severity, calibration Z-scores, and technique-intent matrices. While primarily an application, garak incorporates significant research elements: it implements state-of-the-art attack techniques (GCG, AutoDAN, BEAST, TAP) and includes curated datasets of adversarial prompts and calibration data. The codebase serves both as a practical security tool and as a research platform for LLM vulnerability assessment.
The LLM identified 5 files worth highlighting for reuse:
Defines core data structures (Message, Turn, Conversation, Attempt) for modeling LLM interactions with lifecycle semantics. Self-contained and reusable for any LLM evaluation project that needs structured prompt/response tracking.
Provides a collection of encoding and decoding functions (base64, hex, braille, morse, leetspeak, etc.) used for adversarial prompt obfuscation. Self-contained and reusable for any project requiring text encoding transformations.
Implements ngram containment matching for substring detection with configurable ngram size. A simple but effective pattern for fuzzy matching that can be extracted for use in other detection or search systems.
Computes bootstrap confidence intervals for evaluation metrics with sensitivity/specificity correction. A reusable statistical utility for any evaluation framework that needs robust uncertainty estimation.
Contains curated regex patterns for detecting API keys and secrets from various services (AWS, GitHub, OpenAI, etc.). Self-contained and directly reusable for secret scanning or credential leak detection.