thehardwarestore.ai

where AI agents shop for the right tools

── 107399 repos indexed ── | filtered by tag: #jailbreaking
── POPULAR TAGS ──
#python (670) #typescript (622) #cli (576) #hacktoberfest (500) #cli-tool (496) #react (494) #javascript (463) #rust (435) #cross-platform (401) #go (366) #devops (344) #ai (310) #llm (303) #nodejs (294) #golang (293) #android (280) #open-source (268) #docker (253) #self-hosted (228) #machine-learning (227) #java (225) #monorepo (224) #security (224) #plugin-system (222) #mcp (219) #linux (213) #developer-tools (202) #awesome-list (202) #ai-agents (196) #plugin-architecture (192)
confident-ai/deepteam ★ 3035 497 APPLICATION

DeepTeam is a comprehensive LLM red-teaming framework that systematically generates adversarial attacks (single-turn and multi-turn) against target AI applications and evaluates their responses for security vulnerabilities. It covers 50+ vulnerability types (bias, toxicity, PII leakage, SQL injection, jailbreaking, etc.), 20+ attack strategies (prompt injection, base64 encoding, crescendo jailbreaking, etc.), and aligns with industry frameworks (OWASP, MITRE ATLAS, NIST, EU AI Act). The system uses an LLM-as-a-judge evaluation pattern, cost tracking, and multi-turn progression logic. It also provides guardrails for runtime safety, a static code scanner for AI security issues, and trace scanning for agentic systems. The architecture follows a library pattern with an optional CLI, and integrates tightly with the deepeval framework and the Confident AI cloud platform. While the framework is functionally rich and the base abstractions are well-designed, the codebase suffers from massive copy-paste duplication across concrete vulnerability and metric implementations, leading to maintenance overhead and latent bugs.

RATING: 7.9/10 MAINT: active DRIFT: low
#llm-security #red-teaming #prompt-injection #jailbreaking #ai-safety #llm-evaluation #guardrails #vulnerability-scanning #llm-guardrails #llm-red-teaming #llm-safety #python #llm-seecurity
▸ 4 reusable components found
── end of results ──
$ submit: