Langfuse is an open-source LLM observability and evaluation platform. It provides tracing, prompt management, evaluations, datasets, experiments, monitors, and a public API for integrating with AI applications. The architecture is a monorepo with pnpm workspaces and Turborepo, consisting of a Next.js frontend (web), a background worker (Express + BullMQ), a shared package with types and server utilities, and an enterprise edition. Data flows from ingestion (REST API, OpenTelemetry, SDKs) into ClickHouse for high-volume analytics and PostgreSQL for relational metadata. Background jobs handle evaluations, batch exports, cloud billing, and integrations. The codebase is well-structured with extensive testing, a custom ESLint plugin, and reusable UI components. It is actively maintained with frequent migrations and feature additions.
VoltAgent is a large TypeScript monorepo for building, deploying, observing, and evaluating AI agents. The core package provides an agent runtime with tool calling, memory, subagents, guardrails, workflow orchestration, MCP client integration, A2A protocol support, voice, and OpenTelemetry-based observability. Framework adapters expose this runtime as HTTP/SSE/WebSocket servers via Hono, Elysia, and serverless function providers, while a CLI scaffolds projects and manages VoltOps deployment/auth flows. The repo is fundamentally an application framework, though it contains hybrid data-oriented subsystems: a RAG package with many document chunking strategies, vector adapters, and BM25 search, plus evaluation/scoring packages used for LLM benchmark experiments.
ThinkWatch is a Rust workspace implementing an AI gateway platform that fronts multiple model providers and MCP (Model Context Protocol) servers. It exposes OpenAI-compatible chat/completion endpoints, streaming, MCP tool-call routing, content filtering, and provider failover. The backend is split into crates: `common` (shared lifecycle/limits/access), `auth` (PoW login, API keys), `gateway` (model routing), `mcp-gateway` (MCP server proxying), `server` (REST admin/audit/trace endpoints), and `test-support` (integration test client). A React/TypeScript frontend in `web/` provides an admin console, setup wizard, limits panels, and MCP server management. Data flows: requests enter the gateway, pass through `check_limits`, `check_budget`, `check_access`, then `run_post_invoke`; `trace_id` is propagated across gateway, MCP, audit, and app logs stored in ClickHouse; Postgres holds users, API keys, budgets, and MCP server configs; Redis/Lua scripts enforce rate limits.
agentsre is a Python library that applies Site Reliability Engineering (SRE) practices to AI agent systems. It provides reliability instrumentation for production agents: semantic validation of agent outputs, SLI metrics collection, circuit breaking, escalation baseline detection, cost tracking, and structured audit logging. The codebase is organized as a package with a core reliability module (aos_reliability.py), an orchestration subpackage for control-plane SLIs, a validation layer for agent decisions, and a cost optimizer for LLM spend. It also includes a fintech orchestration demo and documentation for running agents in payment systems. Architecturally, it is a library rather than a service: consumers import the Python API and embed checks into their agent loop. The primary data flow is agent action -> semantic validation -> reliability/risk checks -> orchestration SLI recording -> audit log. The system tracks metrics like escalation rates and decision quality, and can block or allow requests via a circuit-breaker style interface. External dependencies are minimal, with optional cost-optimizer imports guarded by try/except. Maintenance is active, with CI workflows, tests, and a changelog. The code is well-documented and the modules are mostly self-contained, making several components extractable for reuse in other agent platforms.