Presidio is a production-grade PII detection and anonymization framework. It operates as both a Python library and a set of microservices (Flask REST APIs) for detecting and redacting personally identifiable information (PII) in text, images (including DICOM), and structured/tabular data. The system is built around three core engines: the Analyzer (entity recognition via regex patterns, NLP models, and LLMs), the Anonymizer (applies operators like replace, redact, mask, hash, encrypt), and the Image Redactor (OCR-based text extraction and bounding-box redaction). A CLI tool and a structured data engine complement the main services. The architecture is modular and extensible: recognizers are pluggable (predefined regex-based, NLP-based, or custom), anonymization operators follow a consistent interface, and NLP engines (spaCy, Stanza, Transformers, GLiNER, HuggingFace) are swappable via YAML configuration. The system supports context-aware detection, score thresholds, allow/deny lists, and batch processing. External dependencies include NLP libraries, OCR (Tesseract), cloud services (Azure AI Language, Azure Health Data Services), and LLM providers (Azure OpenAI, Ollama). Key entry points include the AnalyzerEngine.analyze() method, AnonymizerEngine.anonymize(), the REST endpoints (/analyze, /anonymize, /redact), and the presidio-cli command-line tool. The primary data flow is: input text/image → Analyzer (detects entities with scores) → Anonymizer (applies operators) → output. For images, OCR extracts text before analysis. The system is designed for compliance with data privacy regulations (GDPR, HIPAA, etc.) and is used in production deployments with Kubernetes, Azure, and Spark.
LiteLLM is a production-grade, open-source proxy/gateway for over 200 large language model (LLM) providers, including OpenAI, Anthropic, Azure, Bedrock, Gemini, and many more. It presents a unified OpenAI-compatible API, handling request routing, authentication, rate limiting, cost tracking, caching, guardrails (content filtering, PII detection), and audit logging. The system is architected as a FastAPI Python server with a Next.js admin dashboard, Rust extensions for performance-critical paths (token counting, caching, auth, cost calculation), and optional PostgreSQL/Redis backends. It supports streaming, MCP (Model Context Protocol), A2A (Agent-to-Agent) protocol, and enterprise features like SSO, secret management, and Terraform deployment modules. Key entry points include the proxy server (litellm/proxy/proxy_server.py), CLI commands (litellm, litellm-proxy), and the admin UI at /ui/litellm-dashboard. The primary data flow is from client requests (chat completions, embeddings, audio, images) received by the proxy, which authenticates, applies guardrails, routes to the configured LLM provider(s), logs usage, and returns a standardized response. Caching is multi-tier (Redis, disk, S3, GCS, Azure Blob, semantic). Spend tracking is aggregated into daily tables and exposed via the management API and UI. External dependencies include PostgreSQL (via Prisma ORM), Redis (for distributed caching, rate limiting, and queuing), and cloud-specific secret managers (AWS Secrets Manager, GCP Secret Manager, Azure Key Vault, HashiCorp Vault). The project uses uv for Python dependency management, Cargo for Rust crates, and npm/pnpm for the dashboard. Extensive CI/CD pipelines (CircleCI, GitHub Actions) run unit, integration, e2e, mutation, and Rust-Python parity tests.
This repository implements a production-grade semantic router for LLM inference, functioning as an Envoy ext_proc filter that intercepts chat completion requests, classifies them via multiple signals (intent, PII, jailbreak, domain, embeddings, etc.), and routes to the most appropriate model or model composition. The system supports Mixture-of-Models (MoM) with configurable algorithms (fusion, cascading, workflows) and provides advanced features like semantic caching, context compression, rate limiting, and observability. It includes a full-stack dashboard (React + Go backend) for configuration management, evaluation/experimentation (campaigns, A/B tests, capacity SLOs), and monitoring. A Python CLI and Kubernetes operator are available for deployment. The codebase demonstrates high engineering quality with extensive validation, testing, and a modular architecture, though its polyglot nature (Go, Rust, Python, TypeScript) and numerous optional backends introduce maintenance complexity. While research elements (training scripts, benchmarks) exist, the primary character is a deployable application.
DeepTeam is a production-oriented Python library and CLI for red-teaming LLM applications. It does not merely document vulnerabilities; it executes automated adversarial campaigns against a user-supplied model callback, generating attack prompts through LLM-driven templates (single-turn, multi-turn, tree/crescendo jailbreaks), scoring target responses with LLM-as-judge metrics, and aggregating outcomes into risk assessments with CVSS scoring. It also ships a code scanner for static AI-security review, a trace scanner for agent executions, and deployable input/output guardrails. Main entry points are the red_team() Python API and the deepteam CLI (login, run, scan), with YAML configuration driving model selection, vulnerability types, and attack budgets. Architecturally, the repo is a modular Python package built on deepeval: vulnerabilities inherit from BaseVulnerability, all metrics inherit from BaseRedTeamingMetric, attacks implement configurable templates and schemas, and frameworks (OWASP, MITRE ATLAS, NIST, EU AI Act, BeaverTails, Aegis) map risks to pre-built categories. Results flow through RTTestCase objects and can be saved locally as JSON or pushed to the Confident AI platform. The library is dual sync/async and includes progress reporting, cost accumulation, and retry logic. Although it embeds research-style synthetic data generation and dataset-driven framework integrations, the dominant character is an application library/CLI rather than a data research project. Strengths include broad vulnerability and attack coverage, clear base-class contracts, a substantial test suite, and active CI. Weaknesses include heavy copy-paste duplication across 30+ vulnerability, metric, and template modules, tight coupling to deepeval internals, reliance on hardcoded model defaults such as gpt-3.5-turbo-0125, and a known logic bug where is_successful in the metric base class compares instead of assigns, always returning False unless an error is set. Telemetry code also contains hardcoded service keys. Overall, it is a functional, well-documented framework with real reuse value, but it would benefit from deduplication and dependency hardening.
Agentspan is a comprehensive, multi-language agent orchestration platform built on top of Orkes Conductor. It provides SDKs in Python, TypeScript, Java, and C# for defining, running, and deploying AI agents with rich features: tools, guardrails, multi-agent strategies (handoff, sequential, parallel, router, swarm, plan-execute), streaming, human-in-the-loop, code execution in sandboxes, credential management, and scheduling. The architecture follows a server/client pattern where agent definitions compile to Conductor workflows, with a Java/Spring Boot server, a Go CLI with a Bubble Tea TUI, and a React/Material-UI web interface for visual workflow editing and execution monitoring. The repository is archived (read-only) as of August 2026, with its active development having merged into the Orkes Conductor repository. Despite being archived, the codebase represents a mature, well-crafted multi-agent runtime with extensive examples, comprehensive testing, and clean abstractions across all supported languages. The SDKs demonstrate consistent design patterns—agent composition via strategies, tool registration via decorators/annotations, guardrail chains with composable termination conditions, and thread-safe credential injection—making the codebase a valuable reference for building agent orchestration systems.