$ show luml-ai/luml
LUML is an agent-native MLOps platform that orchestrates the full machine learning lifecycle: experiment tracking, model packaging, registry, deployment, monitoring, and AI-mediated code generation. The monorepo contains multiple services: a FastAPI backend (multi-tenant platform API with PostgreSQL), a Vue 3 frontend (SPA with Pinia, PrimeVue), a satellite runtime for edge deployment and monitoring (Docker/Kubernetes), an SDK for experiment tracking and model packaging (supporting sklearn, XGBoost, LangGraph, LLM evaluation), a Prisma service for agent-driven code generation (Claude/Codex orchestrator with git worktree isolation), a local experiment tracker (lumlflow) with both a Textual TUI and web interface, and WASM-based in-browser ML workers (prompt optimization, tabular forecasting). The architecture is microservice-oriented with a shared SDK and wire contracts. The platform is production-grade with comprehensive testing (unit, integration, E2E), CI/CD pipelines, OpenAPI contracts, and a well-defined RBAC permission matrix. It demonstrates high craftsmanship in areas like monitoring drift computation (PSI, multivariate), WebWorker RPC proxies, and LLM judge scoring, though the sheer size and number of packages introduce some coordination complexity.
The LLM identified 4 files worth highlighting for reuse:
Defines an abstract hierarchy (`Metric`, `MetricInput`, `MetricComputation`) for computing monitoring metrics such as feature drift, output drift, data quality, and runtime health. The base classes encapsulate the compute-then-threshold pattern with a registry for discoverability. This pattern is reusable in any monitoring system that needs to add custom metrics without modifying the framework.
Implements `ForkNodeHandler`, which reads experiment proposals from multiple file sources (`.proposals/`, `.luml-fork.json`, `.prisma/fork.json`) to automatically branch and spawn child experiment runs. The handler cleanly separates proposal parsing, validation, and node creation, demonstrating a robust design for agent-driven iteration that could be extracted for any automated A/B experimentation system.
Provides `LLMJudgeScorer` and `SupervisedLLMJudgeScorer` abstract classes that use a configurable LLM to evaluate model outputs. Includes OpenTelemetry tracing, automatic retry on parse failures with corrective reminders, and a name disambiguation mechanism for multiple scorers. This base is cleanly separated from concrete scorers (correctness, relevancy, etc.) and can be reused to define custom LLM-judge scorers in any project.
Implements a generic request-response proxy over a Web Worker using unique message IDs, supporting `AbortSignal` cancellation and a lifecycle `init` method. It provides a typed interface for the frontend to communicate with an experiment snapshot worker without exposing worker internals. This pattern is directly reusable for any single-purpose Web Worker communication (e.g., in-browser ML, data processing) and demonstrates clean separation of concerns between UI and worker logic.