$ show osintbrazuca/osint-brazuca-regex
OSINT Brazuca Regex is a CLI tool and curated dataset of regular expressions for detecting Brazilian and global PII/OSINT-relevant patterns. The scanner (`osint_regex_scan.py`) applies the dataset to files, directories, or stdin, supporting multi-threading, text extraction from documents (PDF, DOCX, XLSX, RTF, EPUB) via stdlib or optional `pypdf`/`pdftotext`, and noise filtering (blob masking, line length limits). Output formats include terminal, CSV, JSON, or raw values. The dataset is split into two JSON files (Brazil-specific and general), merged at load time, and includes patterns for CPF, CNPJ, credit cards, crypto wallets, API credentials, and more. The repository also includes a build script (`build.py`) for managing the split/merged dataset. Core functionality requires only Python 3.12+ stdlib, with optional dependencies for extended PDF support and encoding detection.
The LLM identified 2 files worth highlighting for reuse:
The functions for content type detection (`detect_content`, `sniff_path`) and text extraction from documents (`extract_pdf`, `extract_rtf`, `extract_zip_document`) are well-isolated, using only stdlib for most formats and gracefully falling back to external tools. They can be extracted as a reusable 'bin2txt' utility for any project needing to extract text from binary documents before regex scanning.
The build script manages the split/merge lifecycle of the regex dataset, enforcing category classification and integrity checks (no duplicate regexes, compilable patterns). The merge logic correctly handles overlapping categories between the BR and geral files by combining entries rather than overwriting. This pattern is directly reusable for any multi-source JSON dataset that needs to be merged with conflict resolution.