$ show TracyWang95/DataInfra-RedactionEverything
This is a document redaction and anonymization platform designed for processing unstructured data (documents, images, PDFs) and structured data (CSV, databases) to detect and redact sensitive information like PII, PHI, and financial data. The system features a React/TypeScript frontend with a Python/FastAPI backend, utilizing AI models for NER (HaS Text), OCR (PaddleOCR-VL), and visual detection (LocateAnything). It provides both single-file processing via a playground interface and batch processing workflows with review/approval steps. The platform includes role-based access control, audit logging, configurable presets for different industries, and structured data handling capabilities. Deployment is containerized with Docker and supports GPU acceleration for ML models.
The LLM identified 5 files worth highlighting for reuse:
Orchestrates the redaction workflow by coordinating OCR, NER, and visual detection services. Demonstrates clean separation of concerns with dependency injection, making it reusable as a standalone workflow engine for multi-step AI pipelines.
Provides a unified interface for structured data operations (profiling, masking, export) across different database types. Shows excellent encapsulation of complex data source handling with pluggable connectors, suitable for extraction as a reusable data integration layer.
Manages the computer vision pipeline including object detection, region merging, and specialized detectors (QR codes, seals). Exemplifies a well-structured pipeline with clear stage separation and error handling, making it adaptable for other computer vision workflows.
Implements a sophisticated state machine for the 5-step batch processing workflow with persistence to localStorage. Shows excellent React hook design for complex multi-step processes that could be reused in other wizard-style interfaces.
Implements a flexible rate limiting system with multiple algorithms (fixed window, sliding window) and Redis backend. Demonstrates production-ready rate limiting with proper header responses and exemptions, suitable for extraction as reusable middleware.