AIBrix is a Kubernetes-native control plane and intelligent gateway for serving production LLM workloads. The repository implements two intertwined planes: a data plane composed of Envoy ext_proc gateway plugins (cmd/plugins, pkg/plugins/gateway) that intercept OpenAI-compatible inference requests and route them to vLLM, SGLang, or TensorRT-LLM replicas using algorithms such as least-load, prefix-cache-aware routing, SLO queueing, token-load tracking, and prefill/decode disaggregation; and a control plane of Kubernetes controllers (cmd/controllers) that reconcile custom resources for autoscaling (PodAutoscaler), LoRA adapters, model warmup/claims, KV caches, RayCluster fleets, and StormService/RoleSet orchestration. Supporting runtime code includes a Python metadata and batch-job service, a distributed KV-cache package, a GPU optimizer, and an OpenAI-compatible frontend.
Raif (Ruby AI Framework) is a Rails engine/gem that provides a comprehensive, opinionated framework for integrating large language models (LLMs) into Rails applications. It abstracts over multiple LLM providers (OpenAI, Anthropic, Google Gemini, AWS Bedrock, xAI, OpenRouter) with a unified interface for chat completions, streaming, tool calling, structured outputs, and batch inference. The system persists every LLM interaction as `ModelCompletion` records with full token/cost accounting, supports conversation management with tool call validation and retry loops, and provides a task system for running parameterized LLM prompts with caching and temperature control. A key differentiator is its built-in evaluation framework (`Raif::Evals`) that enables dataset-driven evals with LLM-as-a-judge, statistical comparison (bootstrap CI95, sign test, Fisher exact), and regression gating for CI/CD pipelines. Architecturally, Raif is a mountable Rails engine with its own models, migrations, controllers, background jobs, and admin UI. It uses Single Table Inheritance extensively for extensibility (tasks, agents, model completion batches, LLM judges). The system includes a declarative model manifest DSL for defining model capabilities, pricing, and lifecycle, a data retention/archiving subsystem with pluggable storage adapters, and a Prompt Studio for A/B testing prompts across models. The eval framework supports concurrent execution via forked worker processes with per-worker databases, live HTML reports, and resumable run logs. External dependencies include Faraday for HTTP, ActiveJob for background processing, and provider-specific SDKs (AWS SDK for Bedrock). Key entry points include `Raif::Llm#chat` for direct LLM calls, `Raif::Task#run` for task execution, `Raif::Conversation` for multi-turn conversations, `Raif::Agents::NativeToolCallingAgent` for ReAct-style agent loops, and the CLI (`raif evals`, `raif evals:compare`, `raif evals:report`) for running evaluations. The admin web interface provides full CRUD for tasks, conversations, agents, model completions, and batch inference jobs. Background jobs handle archive/cull operations, batch polling, and cost event repair. The system is designed for production use with features like advisory locks for concurrency control, transient retry with exponential backoff, and configurable data retention policies. While primarily an application framework, Raif has strong data/research characteristics through its evaluation framework, model manifest with pricing/capability tracking, and smoke-testing infrastructure for live API verification. The eval framework produces detailed metrics (pass rates, scores, token usage, costs, confidence intervals) and supports comparison reports with regression gates, making it suitable for systematic LLM evaluation in CI/CD workflows. The model manifest serves as a machine-readable catalog of supported models with lifecycle management, enabling automated capability detection and cost estimation.
MegaDetector is a production-grade object detection pipeline specifically designed for camera trap images in wildlife conservation. It detects animals, people, and vehicles in images and videos, outputting bounding boxes with confidence scores in a standardized JSON format (MegaDetector output format). The system provides a comprehensive set of command-line tools for batch inference, post-processing (repeat detection elimination, classification smoothing, format conversion), and integration with platforms like Wildlife Insights and LILA. The architecture is a modular Python package with separate modules for detection (supporting PyTorch and TensorFlow backends), classification, data management, post-processing, visualization, and taxonomy mapping. Key entry points include `run_detector_batch.py` for batch inference, `run_detector.py` for single-image inference, `postprocess_batch_results.py` for generating HTML previews, and `separate_detections_into_folders.py` for organizing results. Data flows from raw images through the detector to JSON results, which are then post-processed (e.g., false positive reduction, confidence thresholding) and optionally converted to other formats (COCO, YOLO, Wildlife Insights). External dependencies include PyTorch, TensorFlow, ultralytics, OpenCV, PIL, and various utility libraries. The repository also includes a C# Windows GUI application for JSON subsetting and integration guides for Timelapse and digiKam.