Presidio is a production-grade PII detection and anonymization framework. It operates as both a Python library and a set of microservices (Flask REST APIs) for detecting and redacting personally identifiable information (PII) in text, images (including DICOM), and structured/tabular data. The system is built around three core engines: the Analyzer (entity recognition via regex patterns, NLP models, and LLMs), the Anonymizer (applies operators like replace, redact, mask, hash, encrypt), and the Image Redactor (OCR-based text extraction and bounding-box redaction). A CLI tool and a structured data engine complement the main services. The architecture is modular and extensible: recognizers are pluggable (predefined regex-based, NLP-based, or custom), anonymization operators follow a consistent interface, and NLP engines (spaCy, Stanza, Transformers, GLiNER, HuggingFace) are swappable via YAML configuration. The system supports context-aware detection, score thresholds, allow/deny lists, and batch processing. External dependencies include NLP libraries, OCR (Tesseract), cloud services (Azure AI Language, Azure Health Data Services), and LLM providers (Azure OpenAI, Ollama). Key entry points include the AnalyzerEngine.analyze() method, AnonymizerEngine.anonymize(), the REST endpoints (/analyze, /anonymize, /redact), and the presidio-cli command-line tool. The primary data flow is: input text/image → Analyzer (detects entities with scores) → Anonymizer (applies operators) → output. For images, OCR extracts text before analysis. The system is designed for compliance with data privacy regulations (GDPR, HIPAA, etc.) and is used in production deployments with Kubernetes, Azure, and Spark.
Hawk-Eye is a command-line application designed to scan various data sources for Personally Identifiable Information (PII) and secrets. It supports cloud storage (S3, GCS, Firebase), databases (MySQL, PostgreSQL, MongoDB, Redis), file systems, and collaboration platforms (Slack, Google Drive). The tool uses regex pattern matching and OCR to detect sensitive data in multiple file formats including images, PDFs, Office documents, and videos. The architecture is modular with a central dispatcher in main.py that routes commands to specific implementations in the commands/ directory. Shared functionality like file reading, OCR, and pattern matching is encapsulated in internals/system.py. This design allows easy extension to new data sources by adding new command modules. The tool is intended for security auditing and can be run standalone or integrated into scripts/CI/CD pipelines. Features include alert deduplication, severity-based reporting, and Slack/Jira integrations. Despite its comprehensive feature set, the tool maintains separation of concerns, making it maintainable and extensible. The use of third-party libraries for specific data sources keeps core logic focused on scanning and reporting.