DeepTeam is a production-oriented Python library and CLI for red-teaming LLM applications. It does not merely document vulnerabilities; it executes automated adversarial campaigns against a user-supplied model callback, generating attack prompts through LLM-driven templates (single-turn, multi-turn, tree/crescendo jailbreaks), scoring target responses with LLM-as-judge metrics, and aggregating outcomes into risk assessments with CVSS scoring. It also ships a code scanner for static AI-security review, a trace scanner for agent executions, and deployable input/output guardrails. Main entry points are the red_team() Python API and the deepteam CLI (login, run, scan), with YAML configuration driving model selection, vulnerability types, and attack budgets. Architecturally, the repo is a modular Python package built on deepeval: vulnerabilities inherit from BaseVulnerability, all metrics inherit from BaseRedTeamingMetric, attacks implement configurable templates and schemas, and frameworks (OWASP, MITRE ATLAS, NIST, EU AI Act, BeaverTails, Aegis) map risks to pre-built categories. Results flow through RTTestCase objects and can be saved locally as JSON or pushed to the Confident AI platform. The library is dual sync/async and includes progress reporting, cost accumulation, and retry logic. Although it embeds research-style synthetic data generation and dataset-driven framework integrations, the dominant character is an application library/CLI rather than a data research project. Strengths include broad vulnerability and attack coverage, clear base-class contracts, a substantial test suite, and active CI. Weaknesses include heavy copy-paste duplication across 30+ vulnerability, metric, and template modules, tight coupling to deepeval internals, reliance on hardcoded model defaults such as gpt-3.5-turbo-0125, and a known logic bug where is_successful in the metric base class compares instead of assigns, always returning False unless an error is set. Telemetry code also contains hardcoded service keys. Overall, it is a functional, well-documented framework with real reuse value, but it would benefit from deduplication and dependency hardening.
nlpaug is a Python library for data augmentation across text (character, word, sentence), audio, and spectrogram modalities. It provides a wide array of augmentation techniques including random insertion/substitution/deletion, synonym replacement via WordNet/PPDB, contextual word embeddings (BERT, GPT-2, RoBERTa), back-translation, TF-IDF-based augmentation, keyboard/OCR error simulation, and audio effects (noise, pitch shift, speed, VTLP). The library follows a consistent pattern: each augmenter inherits from a base class, accepts data via a unified `augment()` method, and returns augmented samples. Composition is supported through `Sequential` and `Sometimes` pipeline classes. It relies heavily on external deep learning models (HuggingFace Transformers, word2vec, GloVe, fastText) and audio processing (librosa). The codebase is well-structured with clear separation of concerns (augmenters, models, utilities, flows), extensive unit tests, and documentation. However, it exhibits significant technical drift due to aggressively pinned dependencies (e.g., `transformers>=5.9.0`, `numpy>=2.4.6`) that may not exist or conflict with current stable releases, and some integration code (conda setup for Python 3.5–3.9) is outdated. Despite this, the core augmentation logic and utility modules are sturdy and reusable.