The OpenAI Agents SDK (Python) is a production-grade library for building agentic AI applications. It provides a complete agent runtime (`Runner`, `Agent`, `RunConfig`) with support for tool calling, handoffs, guardrails, sandboxed execution (Docker, Modal, E2B, Cloudflare, Vercel, Unix-local), realtime voice/audio pipelines, MCP integration, and multi-provider model routing (OpenAI, LiteLLM, AnyLLM, custom). The codebase is async-first with extensive tracing/observability, session persistence (SQLite, OpenAI Conversations, Responses Compaction), and a rich lifecycle-hook system. It includes a large test suite covering unit, integration, and sandbox security scenarios, plus internal tooling skills for release management and code review. The SDK is actively maintained with a 0.23.x version, release automation via release-please, and comprehensive docs in English and Japanese.
Strix is an open-source, AI-powered penetration testing tool that orchestrates multiple LLM-driven agents to autonomously discover and validate security vulnerabilities in web applications, APIs, source code, and infrastructure. The system operates as a CLI application with three interface surfaces: a headless/non-interactive mode for CI/CD pipelines, a feature-rich Go Bubble Tea TUI for interactive scanning, and a local React web viewer for reviewing results. Scans run inside Docker sandboxes with a Caido interception proxy sidecar, allowing agents to execute shell commands, browse applications, modify requests, and apply patches. The architecture follows a multi-agent coordination pattern where a root agent delegates tasks to child agents, tracks coverage, manages budgets, and produces comprehensive reports including CVSS-scored vulnerabilities, proof-of-concept exploits, and fix recommendations. The codebase demonstrates strong engineering practices with extensive test coverage, atomic file operations, output bounding, MCP tool integration, and a sophisticated IPC protocol between the Python backend and Go TUI. Key features include: multi-agent orchestration with parent/child messaging and budget enforcement; Docker sandboxed execution with Caido proxy integration; support for multiple LLM providers (OpenAI, Anthropic, LiteLLM, Bedrock, etc.); MCP (Model Context Protocol) tool integration; SARIF and PDF report generation; cloud platform integration for managed scans; and a local web viewer with live transcript and vulnerability triage. The repository is primarily an application but includes data-oriented components like LLM cost tracking, coverage ledgers, and vulnerability deduplication using LLM-based comparison.
Nanobot is a self-hosted, open-source personal AI assistant framework that operates as a full-stack application. The Python backend provides an agent loop with tool execution, session management, multi-provider LLM orchestration (OpenAI, Anthropic, etc.), and a plugin system for 18+ messaging channels (Telegram, Discord, Slack, WeChat, Feishu, etc.). It exposes a WebSocket-based gateway, an OpenAI-compatible API, and a CLI. Two frontends exist: a rich WebUI built with React/TypeScript/Vite/Tailwind and a terminal TUI built with Bun/OpenTUI. The architecture is event-driven via a message bus, with decoupled channel adapters, provider abstraction, and a config system supporting hot-reload. It includes security features like workspace sandboxing, credential redaction, and SSRF protection. The codebase is extensively tested (hundreds of unit/integration tests) and demonstrates strong engineering practices with clean abstractions, though its large dependency surface and multiple frontends introduce some maintenance surface area.
Neuron AI is a PHP framework for building AI-powered applications with support for multiple large language model providers (OpenAI, Anthropic, Gemini, Mistral, Ollama, AWS Bedrock, etc.), tool integration, structured output, retrieval-augmented generation (RAG), and complex workflow orchestration. It provides a composable agent class that can operate in chat, stream, and structured output modes, a workflow engine with interrupt/resume and persistence, a built-in evaluation framework with LLM-based judges and assertions, and a Model Context Protocol (MCP) client for connecting to external tool servers. The architecture is modular: agents are built on top of workflows, tools are defined via PHP classes with JSON schema generation, and providers are abstracted behind a common interface with traits for shared logic.
Flowcraft is a workflow orchestration framework for building, executing, and monitoring durable workflows. It provides a type-safe, composable API for defining workflows as directed graphs of nodes and edges, supporting both programmatic (fluent) and declarative (JSON) definitions. The framework enables progressive scaling from in-memory execution to distributed systems via adapters for message queues and cloud services, with built-in support for patterns like retries, timeouts, human-in-the-loop, batch processing, and subflows. Key architectural pieces include a dependency-free core runtime, a sophisticated context tracking system for efficient state persistence, a plugin-based adapter architecture for distributed execution, and LLM tooling for workflow introspection and control via natural language.
OpenManus is an open-source framework for building general AI agents, inspired by the Manus agent. It provides a modular architecture with a hierarchy of agents (BaseAgent, ReActAgent, ToolCallAgent, Manus, SWEAgent, BrowserAgent, etc.), a rich set of tools (bash, file editing, web search, browser automation, code execution, chart visualization, desktop automation), and multiple execution modes: single-agent CLI, multi-agent planning flow, MCP client/server, and sandboxed execution via Docker or Daytona. The system is LLM-agnostic, supporting OpenAI, Anthropic, Azure, AWS Bedrock, Ollama, and others through a configurable LLM abstraction. It also implements the A2A (Agent-to-Agent) protocol for inter-agent communication. While primarily an application framework, it includes data analysis and chart generation capabilities, giving it a hybrid character. The codebase is well-structured with clean abstractions, but contains some hardcoded values and Chinese comments that may hinder maintainability.
LlamaIndex is a comprehensive Python framework designed for building Large Language Model (LLM) applications, with a primary focus on Retrieval-Augmented Generation (RAG) and agentic workflows. It functions as a modular library rather than a monolithic application, providing abstractions for data ingestion, indexing, retrieval, and query orchestration. The architecture is split between a core package (`llama-index-core`) containing base interfaces and logic, and a vast ecosystem of integration packages (`llama-index-integrations`) that connect to external LLM providers, vector databases, and data sources. The codebase demonstrates a strong plugin-based design, allowing developers to swap components like embeddings, retrievers, and LLMs without rewriting the core orchestration logic.
Lobster is a local-first workflow runtime for AI agents, implemented as a TypeScript CLI and embeddable SDK. It executes deterministic pipelines defined in JSON, YAML, or .lobster files: steps are shell commands or built-in pipeline commands (where, pick, map, sort, groupBy, table, approve, ask, llm.invoke, openclaw.agent, etc.) connected through typed async-iterator data flow rather than text pipes. Pipelines support approval gates, interactive input, parallel branches, for_each loops, retry/timeout policies, on_error handlers, cost limits, and persistent state that can be suspended and resumed via base64url resume tokens. Architecturally it is a modular Node.js application: a core runtime (src/runtime.ts, src/core/tool_runtime.ts) with a command registry, a workflow file loader/executor, an atomic-file state store, an LLM cost-tracking and accounting layer, and a small SDK (src/sdk/index.ts). CLI entry points are bin/lobster.js, bin/openclaw.invoke.js, and bin/clawd.invoke.js. The package also ships GitHub and Gmail recipes that shell out to the gh and gog CLIs, and can delegate agent turns to OpenClaw. External dependencies are limited to those CLIs, the OpenClaw gateway/LLM adapters, and Node.js >= 22. Notable engineering strengths are crash-safe state persistence, idempotent LLM spend accounting, retry/backoff, abortable process-tree cancellation, and extensive tests covering resume, cancellation, concurrency, and billing edge cases. The repository is unambiguously an application; it has no data-research pipelines, notebooks, or analytics assets. It is a well-maintained, thoughtfully structured codebase with several independently reusable Node.js components.
CrewAI is a large, actively maintained Python framework for orchestrating autonomous AI agents and event-driven multi-agent workflows. The codebase is organized as a monorepo with six libraries under `lib/`: `crewai` (core framework), `crewai-cli` (CLI), `crewai-tools` (tool integrations), `crewai-files` (multimodal file handling), `crewai-core` (shared utilities), and `devtools` (release tooling). The core provides two complementary abstractions — Crews for autonomous role-based agent collaboration and Flows for precise, state-machine-driven orchestration — that can be combined in production applications. The framework supports numerous LLM providers (OpenAI, Anthropic, Gemini, Azure, Bedrock, and many more), a wide array of built-in and third-party tools, MCP/A2A protocol integrations, human-in-the-loop flows, and enterprise deployment via the AMP Suite. The repository has extensive test coverage (pytest with VCR cassettes), strong static analysis (ruff, mypy, bandit), and a well-defined release process. While the framework code is solid and well-structured, the repository is dominated by auto-generated documentation files (thousands of `.mdx` files across multiple versions and languages) and repetitive tool README templates, which inflate the file count but contribute little reusable engineering value.
GPT Researcher is an autonomous research agent that produces comprehensive, cited reports from web and local sources. In production, it operates as a full-stack web application: a FastAPI backend (main.py, backend/server/app.py) exposes REST and WebSocket endpoints for initiating research, streaming progress, and retrieving reports. A Next.js frontend (frontend/nextjs/) provides the user interface, and a lightweight vanilla JS frontend (frontend/scripts.js) is also available. The core Python package (gpt_researcher/) can be used programmatically via the GPTResearcher class or as a CLI tool (cli.py). Architecture follows a planner-executor pattern: given a research task, the system generates sub-queries, executes them in parallel across 20+ search engine wrappers (Tavily, Google, Bing, Brave, DuckDuckGo, ArXiv, PubMed, etc.), scrapes the resulting URLs using multiple backends (BeautifulSoup, browser automation, PyMuPDF, FireCrawl), filters and compresses the context using embedding or keyword-based methods, and finally generates a structured report with citations. The system supports multiple report types (summary, detailed, deep research) and multi-agent orchestrations (LangGraph, AG2) for more complex workflows. Key data flows: user query to sub-query generation (LLM) to parallel search across configured retrievers to URL scraping to context compression to source curation to report generation (LLM) to output as markdown/PDF/DOCX. Real-time progress is streamed via WebSocket. External dependencies include LLM providers (OpenAI, Anthropic, Google, Ollama, etc.), search APIs (Tavily, SerpAPI, etc.), and optional services (MCP servers, vector stores, image generation APIs). The codebase demonstrates strong engineering practices: extensive test suite (200+ tests), plugin architecture for retrievers, configurable context filters, cost tracking, and evaluation frameworks for quality and hallucination. However, it carries maintenance risks from its large dependency surface, multiple parallel agent implementations, and reliance on rapidly-changing external APIs.
OxyGent is a production-grade Python multi-agent system (MAS) framework that unifies LLMs, agents, tools, and flows into a declarative component model. It provides a runtime container (MAS) that orchestrates agent loops (ReAct, Chat, Workflow, PlanAndSolve, Parallel, Reflexion), tool execution (via MCP protocol, FunctionHub decorators, HTTP tools, bank tools), and communication (SSE streaming, A2A protocol for cross-framework interop). The framework ships with a FastAPI web service offering chat, evaluation, prompt management, and SSE endpoints, as well as a built-in web UI. A reference application, OxyBank, demonstrates full-stack data annotation with document ingestion, vector search (Vearch), and annotation agents. The architecture is plugin-based: components reference each other by name, and the MAS container handles lifecycle, wiring, and persistence (Elasticsearch, Redis, Vearch, local fallbacks). OxyGent supports distributed deployments via SSEOxyGent and A2AClientAgent, and integrates with LangChain, LangGraph, AgentScope, and Google A2A SDK. The codebase is well-tested with unit, integration, and regression tests, and includes extensive bilingual documentation.
Agno is a Python framework for building, orchestrating, and deploying AI agents. It provides a unified abstraction over dozens of LLM providers (OpenAI, Anthropic, Gemini, Groq, DeepSeek, etc.) and supports rich tool integration, multi-agent teams, workflow engines, RAG knowledge bases, persistent session storage, and human-in-the-loop patterns. The architecture centers on core classes: `Agent`, `Team`, `Workflow`, `Knowledge`, and `Toolkit`, all composable through a consistent configuration interface. Agents can be served via the `AgentOS` runtime, which exposes REST, MCP, A2A, Slack, and WhatsApp endpoints with JWT-based auth, scheduling, and observability. The framework includes a comprehensive evaluation suite (`AccuracyEval`, `PerformanceEval`, `ReliabilityEval`), a learning machine for persistent user profiling and memory, and an environments module for reproducible agent rollouts and scorer-based export. While primarily an application framework, it also exhibits data/research characteristics through its evaluation infrastructure, knowledge chunking/embedding pipelines, and systematic experiment tooling. The extensive cookbook (300+ examples) doubles as integration tests, demonstrating patterns ranging from simple single-agent tasks to complex distributed teams and nested workflows. Key entry points include `agno.agent.Agent`, `agno.team.Team`, `agno.workflow.Workflow`, `agno.os.AgentOS` (`agno serve`), and the `agno` CLI. The codebase is large and actively maintained, with a broad integration surface across models, vector databases, storage backends, and third-party tools. Technical drift is moderate, with some experimental model wrappers and rapidly evolving model IDs that may require updates.
AgentScope is a full-stack, production-grade multi-agent orchestration platform. The core library (`src/agentscope/`) provides a FastAPI-based agent service with CRUD endpoints for agents, sessions, credentials, channels, knowledge bases, schedules, and SOPs. It features a modular middleware onion pattern for agent lifecycle hooks (reply, reasoning, acting, tool execution), a pluggable storage abstraction (Redis, SQL with Alembic migrations, S3 blob store), and a message bus (in-memory or Redis) for inter-session communication and distributed coordination. The system supports real-time voice (WebRTC, DashScope/OpenAI/Gemini/xAI), RAG pipelines with document parsing and vector stores (Qdrant, Milvus, MongoDB, Elasticsearch), channel integrations (Feishu, DingTalk, Discord), and sandboxed workspace backends (Docker, Kubernetes, E2B, Daytona, Apple Containers, Bubblewrap, OpenSandbox). A full React/TypeScript web UI (`examples/web_ui/frontend`) provides chat, channel management, schedule calendar, knowledge base browsing, and MCP hub browsing. Additionally, a terminal console and a goal-oriented pipeline are provided for developer workflows. The framework emphasizes transparency, controllability, and separation between agent logic and infrastructure, supporting multi-agent collaboration, human-in-the-loop permissions, and long-term memory via ReMe and mem0 middleware.
Goose is an open-source AI agent platform that provides a desktop application (Electron/React), CLI (Rust), and SDK (Python, Kotlin) for interacting with large language models through a unified Agent Client Protocol (ACP). In production, it enables users to define sessions with multiple LLM providers (Anthropic, OpenAI, Google, Azure, Bedrock, local models via MLX/GGUF, and 40+ declarative providers), extend capabilities via MCP servers and built-in platform extensions (shell, image, file editing, code analysis, apps creation), and automate workflows through parameterized recipes. The architecture is a hybrid: a Rust backend handles agent state machine orchestration, tool execution, permission management, and provider abstraction, while an Electron frontend provides a rich chat interface with settings, extension management, scheduling, and MCP apps display. Key entry points include the CLI binary (crates/goose-cli/src/main.rs), the ACP server (crates/goose/src/acp/server/), and the desktop renderer (ui/desktop/src/renderer.tsx). Data flows involve user input → state machine processing → LLM provider calls → tool execution → response streaming back to the UI, with telemetry and cost tracking throughout. External dependencies include OpenAI/Anthropic/Google APIs, MCP servers, Whisper for dictation, and optional services like PostHog and Langfuse for observability.
CowAgent is a production-grade multi-agent AI assistant platform that routes messages from 12+ IM channels (WeChat, Feishu, Telegram, DingTalk, Slack, Discord, QQ, WeCom, web console, desktop Electron app) through a modular agent core. The core implements planning, tool use (bash, file editing, browser, web search, MCP, sub-agent delegation, scheduler, memory, vision), three-tier memory with vector embeddings and markdown-aware chunking, knowledge base management, skill system with SKILL.md manifests and a skill hub, self-evolution (idle-conversation review loop), and multi-agent team orchestration with peer transport. The architecture is a modular monolith: channel adapters normalize messages into a unified ChatMessage, the Bridge dispatches to per-agent/session model resolution, the Agent protocol (Task/AgentResult/AgentAction) drives the execution loop with streaming, cancellation, and steering, and all state persists to SQLite with incremental migration. Key entry points are app.py (startup orchestration), cli/cli.py (daemon lifecycle), channel/web/web_channel.py (HTTP API and SSE streaming), and the desktop Electron client (React/TypeScript with Zustand stores). External dependencies include LLM APIs (OpenAI, Claude, Gemini, DeepSeek, etc.), IM platform webhooks/SDKs, optional voice TTS/ASR providers, and MCP servers. The codebase demonstrates strong engineering practices with extensive regression tests (200+ test files), atomic file writes, thread-safe context propagation via ContextVar, and security guards for credential paths and SSRF.
ChatDev 2.0 (DevAll) is a zero-code multi-agent orchestration platform for building and executing graph-based AI workflows. Users define directed acyclic or cyclic graphs in YAML, composing agent nodes (with LLM providers, memory, tools, thinking modes), human-in-the-loop nodes, Python executors, subgraphs, and control-flow edges with conditions and data transformations. The system supports dynamic parallel execution patterns (Map/Tree), multiple memory stores (simple, file-based FAISS, cloud Mem0), and extensible tooling via local Python functions or MCP servers. The architecture is a full-stack application: a FastAPI backend with REST and WebSocket endpoints for workflow execution, batch runs, artifact streaming, and session management; a Vue 3 frontend with a visual workflow editor (VueFlow), dynamic schema-driven forms, and real-time execution monitoring; a CLI entry point (`run.py`); and a Python SDK (`runtime/sdk.py`) for programmatic use. The codebase is well-structured with registry patterns for extensibility, declarative configuration dataclasses with auto-generated UI forms, and a modular runtime with pluggable node executors, edge conditions, and memory stores. While primarily an application, the project has strong research roots (OpenBMB lab, Tsinghua NLP) and includes academic references (ChatDev, Co-Learning, MacNet). It serves as both a production-grade tool for multi-agent automation and a research platform for experimenting with agent architectures, memory systems, and collaborative workflows. The repository includes 20+ example workflows demonstrating diverse use cases from software development and data visualization to 3D scene generation and deep research.
This is a knowledge/content repository, not a running software system: a curated collection of 11 'Agent Skills' packages compliant with the Agent Skills specification (agentskills.io), designed to be loaded on-demand by AI coding agents (Claude Code, Codex, Cursor, etc.) and other LLM runtimes. Each package is a self-contained directory with a SKILL.md entry point (YAML frontmatter with name/description plus routing instructions), plus references/, assets/, scripts/, and optional agents/ metadata. The skills cover research-analysis (deep-architecture-research with a confirmation-gate and evidence-ledger protocol; neural-mechanism-research; review-ai-conversations with reverse-engineered adapters for Claude Code, Codex, opencode, Grok Build, and Cursor conversation stores; understanding-first-report), software engineering (vibe-coding with scenario/domain/phases sub-references), team collaboration (meeting-coach-leader/worker), agent development (agent-builder with actual Python starter code; skill-judge with a 120-point rubric), and content publishing (media-writer for 7 platforms). There is no application entry point, server, or pipeline; the 'product' is the instructional content and routing logic itself. Although classified as application due to its standalone Python tools and operational guidance (rather than data pipeline), the repo is fundamentally a hybrid: most value comes from the rigor of its written protocols and evaluation rubrics, not executable code. The only real code lives in agent-builder (~4 self-contained Python files: minimal-agent.py, subagent-pattern.py, tool-templates.py, init_agent.py) which implement a clean, security-conscious agent loop. The most impressive craft is the data-engineering-style documentation in review-ai-conversations, which precisely specifies the storage schemas (SQLite tables, JSONL shapes, lineage/parent-child rules) of five proprietary agent tools, each fact tagged with provenance evidence ([CODE]/[DOC]/[ISSUE]) and version-pinned research dates. The skill-judge rubric and deep-architecture evidence protocol likewise demonstrate non-trivial expertise: hard gates, severity levels, evidence-vs-inference discipline, and explicit anti-patterns. The repo is actively maintained (last commit 2026-09-16, in-repo evidence research dated 2026-08-23, plus a documented skill-rename migration path in the READMEs, which exist in English, Chinese, and Japanese). Technical drift is low: the pinned model (claude-sonnet-4-20250514), SDK usage, and cited external tools (Codex 0.148.0, Cursor 2026.08) are current. No test suite exists, but that is the wrong bar for a documentation-oriented collection; the applicable standards are structural validity, evidence quality, and progressive-disclosure design, which are handled well. Weaknesses: media-writer's platform guides are somewhat generic social-media advice, some philosophical prose is repetitive, and the Python examples lack a requirements.txt/pinned dependency file.
AG2 is a production-grade Python framework for building multi-agent AI systems. It provides a unified agent abstraction with support for multiple LLM providers (OpenAI, Anthropic, Gemini, Mistral, Bedrock, Ollama, xAI, Z.AI, DashScope), protocol integrations (A2A, A2UI, MCP, ACP), and sandboxed code execution (Docker, Daytona, local). The architecture is event-driven with a modular middleware pipeline, dependency injection, and a streaming event bus. It includes a full evaluation suite with trace reconstruction, scorers, pairwise comparison, and result persistence. The network layer enables distributed agent coordination via a hub-and-channel model with WAL persistence, identity management, and transport-agnostic envelopes. The framework also supports real-time voice interaction, skills-based toolkits, knowledge stores, and human-in-the-loop workflows. While the surface area is large, the codebase demonstrates high craftsmanship with clean abstractions, protocol-based interfaces, and extensive test coverage across all modules.
Langflow is a full-stack, open-source platform for building and deploying AI-powered agents and workflows. It provides a visual flow builder (React frontend), a Python backend (FastAPI/SQLModel), a CLI executor (LFX), an MCP server for AI tool integration, and a Python SDK. The codebase is a monorepo with multiple packages: the main `langflow` distribution, `langflow-base` (core backend), `lfx` (lightweight executor/CLI), `langflow-sdk` (API client), `langflow-stepflow` (step-based execution), and extension bundles (e.g., lfx-duckduckgo, lfx-arxiv, lfx-ibm, lfx-docling). The backend uses SQLModel with Alembic migrations, Celery for background tasks, FastAPI for REST endpoints, and a plugin/extension system for bundles. The frontend is a React/TypeScript app built with Vite, React Flow, Zustand, and Tailwind. The architecture supports multi-worker deployments, auto-login, JWT auth, rate limiting, MCP server integration, and a rich component library for LLMs, vector stores, tools, and data processing.
Raif (Ruby AI Framework) is a Rails engine/gem that provides a comprehensive, opinionated framework for integrating large language models (LLMs) into Rails applications. It abstracts over multiple LLM providers (OpenAI, Anthropic, Google Gemini, AWS Bedrock, xAI, OpenRouter) with a unified interface for chat completions, streaming, tool calling, structured outputs, and batch inference. The system persists every LLM interaction as `ModelCompletion` records with full token/cost accounting, supports conversation management with tool call validation and retry loops, and provides a task system for running parameterized LLM prompts with caching and temperature control. A key differentiator is its built-in evaluation framework (`Raif::Evals`) that enables dataset-driven evals with LLM-as-a-judge, statistical comparison (bootstrap CI95, sign test, Fisher exact), and regression gating for CI/CD pipelines. Architecturally, Raif is a mountable Rails engine with its own models, migrations, controllers, background jobs, and admin UI. It uses Single Table Inheritance extensively for extensibility (tasks, agents, model completion batches, LLM judges). The system includes a declarative model manifest DSL for defining model capabilities, pricing, and lifecycle, a data retention/archiving subsystem with pluggable storage adapters, and a Prompt Studio for A/B testing prompts across models. The eval framework supports concurrent execution via forked worker processes with per-worker databases, live HTML reports, and resumable run logs. External dependencies include Faraday for HTTP, ActiveJob for background processing, and provider-specific SDKs (AWS SDK for Bedrock). Key entry points include `Raif::Llm#chat` for direct LLM calls, `Raif::Task#run` for task execution, `Raif::Conversation` for multi-turn conversations, `Raif::Agents::NativeToolCallingAgent` for ReAct-style agent loops, and the CLI (`raif evals`, `raif evals:compare`, `raif evals:report`) for running evaluations. The admin web interface provides full CRUD for tasks, conversations, agents, model completions, and batch inference jobs. Background jobs handle archive/cull operations, batch polling, and cost event repair. The system is designed for production use with features like advisory locks for concurrency control, transient retry with exponential backoff, and configurable data retention policies. While primarily an application framework, Raif has strong data/research characteristics through its evaluation framework, model manifest with pricing/capability tracking, and smoke-testing infrastructure for live API verification. The eval framework produces detailed metrics (pass rates, scores, token usage, costs, confidence intervals) and supports comparison reports with regression gates, making it suitable for systematic LLM evaluation in CI/CD workflows. The model manifest serves as a machine-readable catalog of supported models with lifecycle management, enabling automated capability detection and cost estimation.