Yelp/detect-secrets is a production-grade Python library and CLI tool for detecting secrets, API keys, tokens, and other sensitive strings in source code. It operates primarily through a baseline file (`.secrets.baseline`) that records discovered secrets, enabling incremental scanning, pre-commit hook integration, and audit workflows. The tool uses a plugin-driven architecture with over 25 built-in detectors (regex-, entropy-, and keyword-based), plus configurable filters (heuristic, allowlist, word list, gibberish) to reduce false positives. Key operational flows include scanning a codebase to create or update a baseline, running as a pre-commit hook to block new secrets, and an interactive audit mode for labeling secrets as true or false positives. The architecture is modular with clear separation of concerns: plugins define detection logic, filters post-process results, settings manage global configuration, and transformers handle non-standard file formats (YAML, INI). The library also supports network verification for some secret types, automatic baseline upgrades across format versions, and custom plugin/filter injection. It is actively maintained with CI across Python 3.9-3.13 and three operating systems.
Hawk-Eye is a command-line application designed to scan various data sources for Personally Identifiable Information (PII) and secrets. It supports cloud storage (S3, GCS, Firebase), databases (MySQL, PostgreSQL, MongoDB, Redis), file systems, and collaboration platforms (Slack, Google Drive). The tool uses regex pattern matching and OCR to detect sensitive data in multiple file formats including images, PDFs, Office documents, and videos. The architecture is modular with a central dispatcher in main.py that routes commands to specific implementations in the commands/ directory. Shared functionality like file reading, OCR, and pattern matching is encapsulated in internals/system.py. This design allows easy extension to new data sources by adding new command modules. The tool is intended for security auditing and can be run standalone or integrated into scripts/CI/CD pipelines. Features include alert deduplication, severity-based reporting, and Slack/Jira integrations. Despite its comprehensive feature set, the tool maintains separation of concerns, making it maintainable and extensible. The use of third-party libraries for specific data sources keeps core logic focused on scanning and reporting.