$ show Yelp/detect-secrets
Yelp/detect-secrets is a production-grade Python library and CLI tool for detecting secrets, API keys, tokens, and other sensitive strings in source code. It operates primarily through a baseline file (`.secrets.baseline`) that records discovered secrets, enabling incremental scanning, pre-commit hook integration, and audit workflows. The tool uses a plugin-driven architecture with over 25 built-in detectors (regex-, entropy-, and keyword-based), plus configurable filters (heuristic, allowlist, word list, gibberish) to reduce false positives. Key operational flows include scanning a codebase to create or update a baseline, running as a pre-commit hook to block new secrets, and an interactive audit mode for labeling secrets as true or false positives. The architecture is modular with clear separation of concerns: plugins define detection logic, filters post-process results, settings manage global configuration, and transformers handle non-standard file formats (YAML, INI). The library also supports network verification for some secret types, automatic baseline upgrades across format versions, and custom plugin/filter injection. It is actively maintained with CI across Python 3.9-3.13 and three operating systems.
The LLM identified 6 files worth highlighting for reuse:
Defines `BasePlugin` and `RegexBasedDetector` abstract base classes that all secret detectors extend. The `BasePlugin` provides a stable interface (`analyze_line`, `analyze_string`, `should_scan_file`, secret type hints) while `RegexBasedDetector` is a reusable mixin for regex-based detectors, automatically extracting secret values and computing line hashes. This is a clean pattern for adding new detectors with minimal boilerplate.
Implements a global `Settings` singleton that holds plugin, filter, and configuration state. The `transient_settings` context manager allows temporary overrides (e.g., for testing or per-project overrides) without modifying global state. The singleton is lazily initialized and cached via `functools.lru_cache`, providing a thread-safe, immutable-like configuration hub that decouples module-level settings from runtime changes.
Provides a decorator-based dependency injection mechanism for filter functions. Filters declare their required parameters (e.g., filename, line, secret, context) and are called with injected values from the scanning pipeline. This decouples filter logic from the pipeline and makes filters easily testable and reusable across projects. The `injectable_variables` and `self_aware` decorators enable declarative filter registration.
Defines `BaseTransformer` with three abstract methods (`should_parse_file`, `parse_file`, `restore_line`). Transformers convert non-line-based file formats (e.g., YAML, INI) into line-proxies for compatibility with line-oriented secret detection. This pattern isolates format handling and allows easy addition of new transformers (e.g., JSON, TOML) without modifying the core scanning logic.
Handles serialization, deserialization, and automatic upgrades of the `.secrets.baseline` file. It uses a versioned JSON format with separate upgrade modules (`detect_secrets/core/upgrades/`) that migrate old baselines to the current schema. The upgrade logic is decoupled from the main baseline code and can be extended per version. This ensures backward compatibility and safe incremental scanning across tool versions.
A lightweight dependency injection utility that resolves function parameters by inspecting type hints and variable names. It is used to inject plugin and filter dependencies at runtime, reducing coupling between components. The design is self-contained (no external DI framework) and could be extracted as a general-purpose DI helper for any Python project.