Mastra is a TypeScript framework for building AI agents and agentic workflows, delivered as a large multi-package pnpm/turbo monorepo. The code (not the README) shows a production-grade core engine: a Mastra orchestrator class, Agent and DurableAgent implementations, a Workflow/Step execution engine with suspend/resume, scheduling, and evented or durable (Temporal/Inngest) execution, createTool with Zod-schema validation, pluggable storage/vector-store backends (Postgres, LibSQL, DynamoDB, DuckDB, ClickHouse, cloud D1, and more), memory subsystems (working, semantic, observational), RAG pipelines, evals/scorers, and an observability stack emitting traces/metrics to OpenTelemetry, Langfuse, Datadog, Sentry, and others. Around that core sit a Hono-based server with adapters for Express/Fastify/NestJS/Koa, a CLI (start/dev/deploy/traces/codemod), a React Studio/Playground app with a full token-driven design system, client SDKs for JavaScript, React (TanStack Query hooks), and AI SDK stream compatibility, plus auth providers, chat channels, browser-automation packages, voice providers, and deployers for Cloudflare/Vercel/Netlify/sandboxes. This is fundamentally an application framework, though it has data-adjacent subsystems (RAG, evals, vector search, observational memory) that make it lightly hybrid. Architecturally, the monorepo is organized into clearly bounded packages: packages/core, packages/server, packages/cli, packages/playground, packages/playground-ui, packages/mcp, packages/rag, packages/memory, packages/evals, stores/, voice/, auth/, channels/, deployers/, work-spaces/, client-sdks/, and a separate product line under mastracode/ (an agentic coding TUI, web app, factory UI, and headless SDK). The dominant internal patterns are consistent and well-executed: Zod .strict() contracts, a universal createTool boundary, a PlatformProxy abstraction over third-party provider connectors, abstract storage-domain classes implemented by each backend, shared test suites for storage fidelity, MSW-based React testing, and a Storybook-documented design system. Reusable craftsmanship is genuine but uneven: the AI SDK streaming utilities (smooth-stream, sse-heartbeat), the codemod transformer factory, and the browser thread-manager are cleanly isolated and extractable, while auto-generated connector code from divergent upstream templates and four vendored AI SDK versions create documentation drift and maintenance burden. Data flows: user requests enter via server adapters/CLI, reach Agent instances that call LLM providers through model gateways, execute tools (local, MCP, or connected third-party providers), persist messages/threads/traces to configurable stores, and optionally run scored evals over dataset experiments. External dependencies are extensive by design: dozens of LLM providers, multiple database/vector backends, auth identity providers, chat platforms, and observability exporters — all behind pluggable interfaces. The repo is intensely active (bulk daily changesets, automated provider-registry regeneration, renovate dependency updates, zero dependabot alerts) and ships codemods for breaking API migrations, placing technical drift in the low range despite its enormous surface area. Main risks are sheer size, vendored AI SDK copy/paste documentation errors, and behavioral divergence across the proliferation of storage backends.
Gentle-AI is a CLI-driven application that manages AI coding agents through installation, configuration, and review workflows. It provides a deterministic engineering environment with features like persistent memory (Engram™), receipt-driven development (RDD) for verifiable code reviews, and telemetry collection. The architecture is modular, separating concerns into CLI commands, agent adapters, components (like Skills and Context7), review transaction logic, state management, and a text-based user interface. The system interacts with external AI agents (Claude Code, OpenCode, Cursor, etc.) via adapter patterns, manages state in $HOME/.gentle-ai, and supports cross-platform operation (Linux, macOS, Windows). Key entry points include the gentle-ai CLI command and TUI screens for interactive workflows.
Decepticon is an autonomous AI-powered red team agent platform for offensive security engagements. It orchestrates multiple LLM-based agents (recon, exploit, AD, cloud, etc.) via LangGraph to execute complex, realistic attack chains against targets, with strict Rules of Engagement (RoE) enforcement, sandboxed execution, and human-in-the-loop approval. The system provides a web dashboard (Next.js), CLI (Ink), and MCP server for control and monitoring, and includes a plugin system for extensibility. It also features comprehensive benchmark frameworks for evaluating agent performance on security challenges. The architecture is a multi-service Docker Compose deployment with Python backend services, a Go launcher, Neo4j knowledge graph, and high isolation between management and sandbox networks. This is primarily a software application (automated red team orchestrator) with strong data/research elements such as knowledge graph persistence, benchmarking pipelines, and telemetry collection. The codebase demonstrates high craftsmanship with protocol-based plugin contracts, a well-structured middleware stack, and thorough testing, though the polyglot nature and dual client maintenance add complexity.
AxonHub is a production-grade, multi-provider LLM API gateway that routes, transforms, and manages requests to a wide range of AI providers (OpenAI, Anthropic, Gemini, DeepSeek, xAI, and 60+ others). It provides a unified API surface for chat completions, embeddings, moderations, audio, images, and video generation, with sophisticated features like load balancing (latency-aware, weighted round-robin, circuit breaker), per-channel rate limiting, provider quota enforcement, streaming, and reasoning token handling. The backend is written in Go using Gin, Ent ORM, gqlgen, and Uber Fx for dependency injection, supporting PostgreSQL, MySQL, and SQLite with automatic migration. The frontend is a comprehensive React/TypeScript SPA (Vite, TanStack Router/Query, Tailwind, shadcn/ui) that provides administration of channels, models, API keys, users, roles, projects, and system settings. It includes analytics dashboards, request tracing, usage logs, and OIDC SSO. The system is designed for multi-tenancy with fine-grained RBAC (scope-based permissions) and privacy policies enforced at the Ent ORM layer. Deployment is supported via Docker, Docker Compose, and Helm charts. The codebase exhibits strong engineering practices: extensive unit and integration tests (including race-condition tests and E2E Playwright tests), pipeline middleware for request transformation, generic caching abstractions, stream processing utilities, and a Zanzibar-inspired authorization framework. While there are minor issues (duplicate utility packages, a typo in a folder name, and some frontend boilerplate repetition), the overall architecture is clean, modular, and well-documented. The project is actively maintained with frequent commits, issue triage, and a clear release process.
Ollama is a local LLM serving platform written in Go. It provides a REST API (OpenAI-compatible), CLI, TUI chat, and desktop application for running large language models locally. The system supports multiple inference backends: llama.cpp (CPU/CUDA/ROCm/Vulkan) and MLX (Apple Silicon). It includes a model conversion pipeline from HuggingFace safetensors to GGUF format, GPU hardware discovery, and an agent framework with tool execution (bash, file operations, web search). The architecture is a monolith with subprocess management for inference backends. Key entry points include the CLI (main.go), HTTP server (server/routes.go), runner binary (cmd/runner/main.go), and desktop UI (app/ui/app/). The codebase is actively maintained with extensive test coverage and support for dozens of model architectures. While primarily an application, it has significant data/research components in model conversion, tokenization, and speculative decoding research.
LiteLLM is a production-grade, open-source proxy/gateway for over 200 large language model (LLM) providers, including OpenAI, Anthropic, Azure, Bedrock, Gemini, and many more. It presents a unified OpenAI-compatible API, handling request routing, authentication, rate limiting, cost tracking, caching, guardrails (content filtering, PII detection), and audit logging. The system is architected as a FastAPI Python server with a Next.js admin dashboard, Rust extensions for performance-critical paths (token counting, caching, auth, cost calculation), and optional PostgreSQL/Redis backends. It supports streaming, MCP (Model Context Protocol), A2A (Agent-to-Agent) protocol, and enterprise features like SSO, secret management, and Terraform deployment modules. Key entry points include the proxy server (litellm/proxy/proxy_server.py), CLI commands (litellm, litellm-proxy), and the admin UI at /ui/litellm-dashboard. The primary data flow is from client requests (chat completions, embeddings, audio, images) received by the proxy, which authenticates, applies guardrails, routes to the configured LLM provider(s), logs usage, and returns a standardized response. Caching is multi-tier (Redis, disk, S3, GCS, Azure Blob, semantic). Spend tracking is aggregated into daily tables and exposed via the management API and UI. External dependencies include PostgreSQL (via Prisma ORM), Redis (for distributed caching, rate limiting, and queuing), and cloud-specific secret managers (AWS Secrets Manager, GCP Secret Manager, Azure Key Vault, HashiCorp Vault). The project uses uv for Python dependency management, Cargo for Rust crates, and npm/pnpm for the dashboard. Extensive CI/CD pipelines (CircleCI, GitHub Actions) run unit, integration, e2e, mutation, and Rust-Python parity tests.
Ray is a distributed compute framework that unifies task/actor execution, data processing, model training, hyperparameter tuning, RL, and model serving under a single runtime. The core is written in C++ (raylet, GCS, object store, scheduling) with a Python frontend, communicating via gRPC and Arrow-compatible plasma object storage. Production entry points include `ray.init()` for cluster bootstrap, `@ray.remote` for task/actor offload, Ray Serve for model deployment (FastAPI/DAG/gang-scheduling patterns), Ray Data for streaming ETL, Ray Train/Tune for distributed training and hyperparameter search, and RLlib for reinforcement learning. The architecture is a hybrid: a distributed systems platform (scheduler, object store, autoscaler) layered with high-level AI libraries, all built via Bazel with extensive CI (Docker images, build matrices, release tests). It is not a data research repo, though Ray Data and RLlib contain research-grade data pipeline and RL primitives; the dominant character is a production-grade application framework.
LightRAG is a production-ready Retrieval-Augmented Generation (RAG) framework that combines knowledge graph construction with vector-based retrieval. The system implements a dual-layer architecture managing both knowledge graphs (entities, relationships) and vector embeddings, enabling both local (entity-focused) and global (theme-focused) query modes plus hybrid combinations. It provides a FastAPI-based REST server with an embedded React WebUI, supporting 10+ LLM providers (OpenAI, Azure, Anthropic, Gemini, Ollama, Bedrock, etc.), 8+ vector/graph storage backends (PostgreSQL+pgvector/AGE, Neo4j, Milvus, Qdrant, Redis, MongoDB, OpenSearch, FAISS, NanoVectorDB), and a multimodal document parsing pipeline integrating MinerU and Docling for PDF/Office documents with tables, equations, and images. The codebase demonstrates strong engineering practices: async-first architecture with configurable concurrency controls, role-specific LLM configuration (EXTRACT, QUERY, KEYWORDS, VLM), atomic file operations for crash-safe persistence, comprehensive test coverage (300+ tests), and production deployment artifacts (Docker, Kubernetes/Helm, systemd). The parser layer is particularly well-designed with a registry-based engine dispatch (native docx/markdown, MinerU, Docling) producing a unified sidecar format. Recent additions include reranking support, asymmetric embeddings, document deletion with KG regeneration, and RAGAS/Langfuse integration for evaluation and observability.
QMD (Quick Markdown Search) is a local-first, on-device hybrid search engine for markdown files, built as a CLI tool, a TypeScript SDK, and an MCP (Model Context Protocol) server. It indexes markdown documents and supports BM25 full-text search via SQLite FTS5, vector semantic search via sqlite-vec, LLM-based query expansion (using a fine-tuned Qwen3-1.7B model), and cross-encoder reranking. The architecture is a monolithic TypeScript module with clear separation of concerns: a core store layer handling indexing and retrieval, a config-driven collection system with YAML-based index definitions, a trust-gated security model for project-local configurations, and output formatters supporting JSON, CSV, Markdown, and XML. The project also includes a fine-tuning pipeline (Python-based SFT, GRPO) to train the query expansion model, with curated training data and evaluation benchmarks. The repository has a hybrid character: while its primary output is a production-ready application (CLI + MCP server), it contains a substantial data-research component in the `finetune/` directory with training scripts, dataset generation, reward functions, and model evaluation harnesses. The fine-tuning pipeline is tightly coupled to the main application's query expansion feature, making this more than a simple add-on – it is integral to the search quality. The code demonstrates strong engineering practices: content-addressable storage for deduplication, AST-aware chunking for code files, reciprocal rank fusion for multi-signal search, and a cross-runtime SQLite abstraction that runs on both Node.js and Bun. The project is actively maintained with CI/CD on GitHub Actions, Nix flake builds, and npm publishing. Key entry points include `src/cli/qmd.ts` (CLI dispatcher), `src/store.ts` (core data layer), `src/mcp/server.ts` (MCP server), and `finetune/train.py` (training pipeline). The system is designed for personal knowledge management and AI agent integration, offering a `qmd search` command for quick lookups and an MCP server exposing tools like `query`, `get`, and `multi_get` for agent consumption. Security is a first-class concern: project-local configs that declare hooks or out-of-project paths require explicit trust approval via a digest-based store, and the HTTP MCP server includes DNS-rebinding protection.
MockServer is a highly sophisticated, multi-protocol HTTP(S) mock server and proxy for testing, written in Java with Netty. It supports HTTP/1.1, HTTP/2, HTTP/3 (QUIC), gRPC, WebSockets, DNS, TCP, and binary protocols on a single port. The core engine provides expectation-based request matching, response generation, forwarding, proxying, recording, verification, and chaos engineering (latency, connection drops, error injection). It also includes built-in LLM mocking (OpenAI, Anthropic, Gemini, etc.), OIDC/OAuth2/SAML identity providers, SCIM provisioning, WASM custom rules, and drift detection. The repository is a monorepo containing the Java server, multi-language client libraries (Java, Node, Python, Ruby, Go, .NET, Rust, PHP), a React dashboard UI, IDE plugins (JetBrains, VS Code), Helm charts, Docker images, and extensive CI/CD pipelines via Buildkite. The architecture is modular with clear separation of concerns: core model/matcher/serialization in `mockserver-core`, Netty transport in `mockserver-netty`, language-specific clients in `mockserver-client-*`, and a React UI in `mockserver-ui`. The codebase demonstrates strong engineering practices including DTO patterns with Jackson serializers, a matcher hierarchy with compiled-once validators, bounded caches, constant-time equality checks, and extensive test coverage with integration tests across multiple environments.
MarkItDown is a Python library and CLI tool that converts a wide variety of file formats (PDF, DOCX, PPTX, XLSX, HTML, CSV, JSON, EPUB, Outlook MSG, images, audio, YouTube, RSS, IPYNB, ZIP, and more) into Markdown. Its primary production use is preparing documents for LLM ingestion, agent workflows, and text analysis pipelines. The architecture is a monorepo containing the core `markitdown` package, an OCR extension (`markitdown-ocr`) that uses LLM Vision for image text extraction, an MCP server (`markitdown-mcp`) exposing conversion as a tool for AI agents, and a sample plugin for third-party converters. The core library uses a plugin system based on `importlib.metadata.entry_points` to register converters, each implementing a `DocumentConverter` base class with `accepts()`/`convert()` methods and a priority-based dispatch. The main entry points are the `MarkItDown` class (Python API), the `markitdown` CLI, and the MCP server's `convert_to_markdown` tool. Data flows from local files, streams, HTTP responses, or URIs through converter selection (by MIME type, extension, or content inspection) to produce Markdown output. External dependencies include many optional format-specific libraries (pdfminer.six, python-docx, python-pptx, openpyxl, etc.), external binaries (exiftool, ffmpeg), cloud APIs (Azure Document Intelligence, Azure Content Understanding), and an OpenAI-compatible LLM client for image captioning and OCR.
Graphify is a CLI tool and library that extracts structural and semantic knowledge graphs from source code across 36+ programming languages. It parses files using tree-sitter grammars, builds a graph of nodes (functions, classes, modules, etc.) and edges (calls, imports, inheritance, etc.), performs community detection via Leiden/Louvain, and generates reports, wikis, and exports to formats like Obsidian, Neo4j, and FalkorDB. The system integrates with AI coding assistants through an MCP server (stdio/HTTP), skill files (SKILL.md, CLAUDE.md), and git hooks for automatic graph updates. Key entry points include `graphify extract`, `graphify query`, `graphify serve`, and `graphify install`. External dependencies include tree-sitter grammars, LLM APIs (OpenAI, Anthropic, Gemini, Ollama), and optional databases. The architecture is modular: per-language extractors in `graphify/extractors/`, graph assembly in `build.py`, clustering in `cluster.py`, and export in `export.py`. The codebase demonstrates strong engineering practices with extensive tests, atomic file writes, security validation (SSRF, path traversal), and cross-platform support.
TradingAgents is a multi-agent LLM framework for financial trading research and backtesting. It orchestrates specialized agents (analysts, researchers, debaters, trader, portfolio manager) through a LangGraph state machine to produce trading decisions. The system emphasizes point-in-time data isolation to prevent lookahead bias, supports multiple LLM providers and data vendors, and includes a comprehensive backtesting engine. It is structured as a Python library with a rich CLI interface. The architecture is a hybrid of a software application and a research framework. It demonstrates high craftsmanship in its data pipeline (vendor routing, symbol normalization, date windowing), LLM integration (capability registry, structured output fallback), and concurrency handling (atomic file writes). While the core abstractions are clean and reusable, the codebase is tightly coupled to the specific behaviors of its external dependencies (LLM APIs, Yahoo Finance, SEC EDGAR), making it sensitive to ecosystem drift. The extensive test suite provides good coverage for unit and integration scenarios, but the reliance on mocking means regressions in vendor adapters may go undetected without manual integration runs.
Crawl4AI is a production-grade, async-first web crawling and scraping library designed to transform any website into clean, LLM-ready Markdown or structured data. It serves as both a Python library and a self-hosted Docker API server, making it suitable for RAG pipelines, AI agents, and data extraction workflows. The architecture is built on pluggable strategies: extraction (CSS, XPath, LLM, Regex), content filtering (BM25, pruning, LLM), chunking (topic, regex, sentence, sliding window), deep crawling (BFS, DFS, Best-First), and proxy rotation. Key entry points include the `AsyncWebCrawler` class with `arun()`/`arun_many()` methods, configuration objects (`BrowserConfig`, `CrawlerRunConfig`), and a CLI (`crwl`). The data flow progresses from URL → browser/HTTP fetch → raw HTML → cleaned HTML → Markdown → filtered content → structured extraction, with caching and cache validation as first-class features. External dependencies include Playwright for browser automation, httpx/aiohttp for HTTP, lxml for fast parsing, and optional LLM providers (OpenAI, Anthropic, Groq, etc.) and ML models (sentence-transformers). The Docker deployment adds a FastAPI server with JWT auth, Redis-backed job queue, webhook support, MCP bridge, and a monitoring dashboard, all hardened against SSRF, XSS, and file-write attacks. While primarily an application, the repository also contains data-research elements such as adaptive crawling with embedding-based relevance scoring, anti-bot detection heuristics, and content filtering research, reflecting a hybrid character dominated by the application/library use case.
rtk is a Rust CLI proxy and hook system designed to minimize LLM token consumption for AI coding agents by filtering and compressing the output of shell commands. It operates as a command wrapper that intercepts commands like git status, pytest, or cargo test before they reach an agent's context window, runs the real tool, and emits a compact, token-optimized summary while preserving diagnostic value. The tool includes a sophisticated architecture: a core runner with a 'never-worse' guard that ensures filtered output never exceeds raw output in size, tiered parsing (JSON → regex → passthrough) for dozens of ecosystems, a declarative TOML filter DSL compiled into the binary, and a SQLite-based recall store for content-addressed output retrieval. It also features a permission model for hooks that can allow, deny, or ask on command rewrites, and integrates with 18+ AI agents (Claude Code, Copilot, Cursor, etc.) via custom hook formats. The repo is a single Rust binary with subcommands for execution, analytics (rtk gain), and session discovery (rtk discover), supported by an extensive test suite with fixture-based golden tests. While not a data research project, it incorporates data-driven elements like token savings accounting and telemetry, but its dominant character is a high-craft CLI application.
OpenViking is an agent-native context database that unifies memories, resources, and skills under a virtual filesystem paradigm (viking:// URIs). It provides a three-tier context model (L0 abstract, L1 overview, L2 full content), recursive directory-based retrieval with visualized trajectories, and automatic session memory extraction. The system consists of a FastAPI HTTP server with embedded Rust components (RAGFS filesystem, vector index engine), a Rust CLI (ov), Python SDKs, a React/Vite Web Studio dashboard, and VikingBot - an agent framework with multi-channel support (Telegram, Slack, Feishu, etc.) and MCP integration. It integrates with OpenClaw, Claude Code, Codex, OpenCode, Hermes, and LangChain/LangGraph. The codebase demonstrates strong engineering: clean separation of concerns, async-first design, comprehensive test coverage (unit, integration, e2e), and production-ready deployment artifacts (Docker, Helm, multi-arch builds). Benchmarks show significant improvements over baseline agents on LoCoMo (82% vs 24-57% accuracy), tau2-bench (+6-12pp), and HotpotQA (91% at top-20).
llmfit is a comprehensive LLM model analysis and recommendation system that matches models to local hardware specifications. It detects system capabilities (CPU, RAM, GPU/VRAM), analyzes model requirements across multiple dimensions (memory fit, speed, quality, context), and provides recommendations for optimal model selection. The system features multiple interfaces including a terminal user interface (TUI) as the default mode, command-line interface, embedded web dashboard, and REST API. It supports various inference backends (Ollama, llama.cpp, MLX, Docker Model Runner, etc.), includes benchmarking capabilities to measure real-world performance, and enables users to share benchmark results with the community. The architecture follows a modular Rust workspace design with llmfit-core as the shared library containing hardware detection, model analysis, and provider abstractions, while interface-specific concerns are separated into llmfit-tui, llmfit-desktop, llmfit-web, and llmfit-python packages.
Langfuse is an open-source LLM observability and evaluation platform. It provides tracing, prompt management, evaluations, datasets, experiments, monitors, and a public API for integrating with AI applications. The architecture is a monorepo with pnpm workspaces and Turborepo, consisting of a Next.js frontend (web), a background worker (Express + BullMQ), a shared package with types and server utilities, and an enterprise edition. Data flows from ingestion (REST API, OpenTelemetry, SDKs) into ClickHouse for high-volume analytics and PostgreSQL for relational metadata. Background jobs handle evaluations, batch exports, cloud billing, and integrations. The codebase is well-structured with extensive testing, a custom ESLint plugin, and reusable UI components. It is actively maintained with frequent migrations and feature additions.
Composio is a production-grade, dual-language SDK monorepo that connects AI agents to external services (Gmail, GitHub, Slack, Notion, Salesforce, and hundreds more) through a unified tool abstraction. In production it operates as the client layer of the Composio platform: applications create a Composio session bound to a user_id, discover toolkits, manage OAuth auth-configs and connected accounts, and execute tool calls either remotely or via a session-scoped Tool Router that also exposes hosted MCP endpoints. The repository ships a TypeScript SDK (@composio/core), a Python SDK (composio), a standalone Effect-TS-based CLI, 15+ framework provider adapters (OpenAI, Anthropic, LangChain, LlamaIndex, CrewAI, AutoGen, Mastra, Vercel AI, Google/ADK, etc.), and a Next.js documentation site. It is unambiguously an application/developer-platform repo rather than data research, although it contains data-adjacent engineering assets: a cross-SDK JSON Schema conversion corpus, OpenAPI dereferencing tooling, a knowledge-base hybrid-search index, and telemetry redaction utilities.
Qwen Code (formerly Gemini CLI) is an open-source, multi-platform AI coding assistant designed to operate in the terminal, web shell, desktop (Tauri), VS Code, JetBrains, and chat channels (DingTalk, WeChat, Slack, etc.). The architecture is a monorepo with ~30+ packages centered around a core library (`packages/core`) that provides agent runtime, tool orchestration, memory, MCP support, and telemetry. The primary entry points are the CLI (`packages/cli` with an Ink-based TUI), the web shell (`packages/web-shell`), and a daemon server (`packages/cli/src/serve/`) that exposes REST/SSE endpoints for session management, file editing, and sandboxed shell execution. The system supports multiple LLM providers (OpenAI, Anthropic, Gemini, DeepSeek, etc.) and features sub-agents, goal-oriented workflows, approval modes, browser automation (CUA driver in Rust), and a Java managed-agent server for enterprise deployments. Data flows are user-to-agent: user input enters via a client (CLI, web, IDE), is processed by the core agent loop, which invokes tools (shell, file system, LSP, MCP servers) and streams responses back. External dependencies include Node.js v22+, pnpm, and platform-specific native binaries (Rust CUA driver, Electron desktop). The codebase is actively maintained with daily commits, extensive integration tests, and CI/CD pipelines. While it is a software application, it includes telemetry modules and a few Python scripts for benchmark analysis, but no research-grade data pipelines or ML training code.