The OmniParser repository provides a system for parsing GUI screenshots into structured elements (text and icons) to enable vision-based GUI agents. It combines object detection (YOLO), OCR (PaddleOCR), and vision-language models (Florence2) to detect and describe screen elements, outputting bounding boxes and structured data. The system includes a FastAPI server for parsing as a service, Gradio and Streamlit demos for interactive use, and agent frameworks that integrate with LLMs (like GPT-4o, Claude) to perform automated computer tasks within a Windows VM environment (via the OmniBox component). The architecture is modular, with core parsing utilities in the `util` directory, agent tools and loops in `omnitool/gradio`, and VM management in `omnitool/omnibox`. Key reusable components include the model loading and processing utilities (`util/utils.py`), a tool abstraction pattern for defining agent actions (`omnitool/gradio/tools/`), and a visualization utility for annotating screenshots (`util/box_annotator.py`). The system demonstrates non-obvious skill in integrating multiple ML models for comprehensive screen understanding and providing a standardized interface for agent-LLM-tool interaction. While actively maintained (last commit July 2026), the repository shows signs of technical drift, including dependency version pinning that may become outdated and deprecation warnings in the demo notebook (e.g., timm layers, Florence2 GenerationMixin). The reliance on external services (Hugging Face models, LLM APIs) and the complexity of the Windows VM setup also introduce maintenance considerations. Nonetheless, the codebase delivers on its promise of enabling vision-based GUI agents and offers several patterns reusable in similar projects.
This repository is the engineering backbone for llamafile, a project by Mozilla.ai that packages large language model (LLM) inference engines into single, portable executables. It is primarily an application distribution and runtime platform, not a data research project. The core innovation is the use of Cosmopolitan Libc to create "Actually Portable Executables" (APEs) that run on Linux, macOS, Windows, and BSD without installation. The codebase integrates heavily patched versions of llama.cpp, whisper.cpp, stable-diffusion.cpp, and agent.cpp as submodules. It provides a unified CLI, TUI (chatbot), and HTTP server (OpenAI-compatible API) for interacting with these models. It features a sophisticated dynamic GPU backend loader (CUDA, ROCm, Vulkan, Metal), a custom high-performance CPU matrix multiplication library (TinyBLAS), integer quantized kernels (IQK), and a cross-platform sandboxing module (pledge/SECCOMP/Landlock). An agent framework (agentfile) allows for tool-calling workflows. The codebase demonstrates high engineering craftsmanship in specific areas, particularly the TinyBLAS CUDA replacement, the dynamic GPU backend abstraction, the state-machine syntax highlighting library, and the embedded readline replacement (Bestline). However, the project carries significant technical drift risk due to its heavy reliance on fast-moving upstream submodules and a complex, platform-specific build system. The patch management workflow is well-documented but inherently fragile.
Youtu-Agent is a comprehensive AI agent framework developed by Tencent Cloud's Youtu Lab, designed for building, orchestrating, and evaluating LLM-powered agents. It provides a modular toolkit system, multi-agent orchestration patterns (simple, orchestra, orchestrator, workforce), a reusable configuration system (Hydra+YAML), and support for various environments (local shell, Docker, E2B sandbox, browser). The framework also includes a research-grade evaluation pipeline with benchmarks like GAIA, SWE-bench, WebWalkerQA, and a training-free GRPO module for agent self-improvement through experience distillation. Two front-end applications accompany the backend: a React-based chat UI and a Next.js evaluation analysis dashboard, both communicating via WebSocket and REST APIs. While the primary character is an application framework, Youtu-Agent has a strong data/research dimension: it ingests benchmark datasets, runs model-based judges, stores trajectories in SQL/PostgreSQL, and implements a meta-learning loop that extracts 'experiences' from agent behaviors. This hybrid nature makes it suitable both for production agent deployment and for conducting RL/benchmarking experiments. The codebase is well-organized into distinct modules (agents, tools, config, eval, practice, tracing, utils), uses modern Python packaging (uv, pyproject.toml), and includes extensive example configs and scripts. External dependencies include the openai-agents SDK, various LLM providers (OpenAI, Anthropic, etc. via litellm), search APIs (Serper, Jina, Google), document parsers (Chunkr, Unstructured), and MCP servers. Deployment supports Docker and multiple environment types. The repository is actively maintained with CI/CD, documentation (mkdocs), and a growing set of features.
AgentFlow is a framework for building and training agentic LLM systems using reinforcement learning, specifically Flow-GRPO (Flow-based Group Refined Policy Optimization). It provides a modular architecture with four core roles—Planner, Executor, Verifier, and Generator—that coordinate via evolving memory and tool calls over multiple turns. The system supports both inference (via a Solver) and distributed RL training (via a Trainer/Client/Server architecture built on verl, vLLM, and Ray). It includes a multi-provider LLM engine abstraction (OpenAI, Anthropic, Gemini, DeepSeek, etc.), a tool system with metadata-driven discovery, and instrumentation for tracing and reward collection. The repository is heavily research-oriented, with benchmark datasets (AIME24, GPQA, GAIA, HotPotQA, etc.) and training pipelines for optimizing agent planners. While primarily an application framework, it has strong data/research components, making it a hybrid project with a focus on reproducible agentic reasoning experiments.
ProxyAI (formerly CodeGPT) is a JetBrains IDE plugin that provides AI-powered code assistance including chat, inline editing, code completions, agent-based task execution, commit message generation, and next-edit suggestions. The plugin integrates with multiple LLM providers (OpenAI, Anthropic, Google, Mistral, Ollama, llama.cpp, custom OpenAI-compatible, and its own ProxyAI cloud) and supports local inference via a bundled llama.cpp submodule. It implements an agent framework with a tool system (read, write, edit, bash, web search, diagnostics, debugger control), MCP (Model Context Protocol) client for external tool servers, and ACP (Agent Client Protocol) for external agent integration. The architecture follows IntelliJ Platform conventions: services, actions, tool windows, settings via PersistentStateComponent, and editor listeners. The codebase is a mix of Kotlin (dominant) and Java, with Gradle build system and extensive use of coroutines for async operations.
Elysia is an open-source agentic platform that uses a decision tree architecture to dynamically select and execute tools for searching and reasoning over vector databases (Weaviate). The core is a `Tree` class that maintains conversation history, an environment of retrieved objects, and a decision loop: at each step, an LLM (via DSPy) chooses which tool to call based on the current state and user goal. Built-in tools include hybrid/semantic query, aggregate, summarization, visualisation, and linear regression; custom tools can be added via a `@tool` decorator or by subclassing `Tool`. The system preprocesses Weaviate collections to generate LLM-created summaries, field mappings, and statistics, enabling adaptive query formulation. The platform includes a full-stack web application: a FastAPI backend with WebSocket endpoints for real-time query execution, and a built Next.js frontend (served as static files) offering a UI for conversations, collection management, tree visualisation (React Flow), and feedback collection. The backend manages multi-user and multi-conversation state via `UserManager`/`TreeManager`, persists trees to Weaviate, encrypts API keys, and supports feedback-driven few-shot learning. A CLI entry point (`elysia start`) launches the API server.
Agentic Security is an open-source LLM vulnerability scanner that probes AI endpoints for security weaknesses such as jailbreaks, prompt injection, data leakage, and sandbox escapes. It provides both a FastAPI web server (port 8718) with a Vue.js frontend and a Fire-based CLI (server, ci, init, ls, scan). The core scan engine sends adversarial prompts (text, image, audio) to target LLMs via configurable HTTP specs, classifies responses using a multi-layered refusal detection system (phrase-based, OneClassSVM, PII detector, LLM-as-judge, hybrid voting), and generates detailed reports with failure rates and cost metrics. The architecture is modular: LLM provider abstraction (OpenAI, Anthropic, LiteLLM), resilience primitives (rate limiter, circuit breaker, concurrent executor), plugin-based refusal classifiers, and a dataset registry supporting HuggingFace, CSV, Google Sheets, and dynamic transformations (stenography, encoding). It also integrates external red-teaming tools (Garak, Inspect AI) and includes reinforcement learning and Bayesian optimization for adaptive prompt selection. While primarily an application, it contains significant data/research components (ML models, datasets, refusal classification research) that support its scanning mission.
This repository is a Python client SDK for the OGX AI inference API, which is OpenAI-compatible. It provides both synchronous (`OgxClient`) and asynchronous (`AsyncOgxClient`) clients with typed resources for chat completions, embeddings, vector stores, models, prompts, responses, batches, files, and more. The SDK is auto-generated by Stainless from an OpenAPI spec, ensuring the API surface stays aligned with the backend. It includes a CLI tool (`ogx-client`) for interacting with the OGX server, and a higher-level agent library (`lib/agents/`) for building conversational agents with tool use, turn/step event models, and ReAct-style reasoning. The architecture follows a resource pattern: each API endpoint group (e.g., `chat/completions`, `vector_stores`) is a class inheriting from `SyncAPIResource`/`AsyncAPIResource`, registered as properties on the client. The base client handles HTTP transport via `httpx`, retries with exponential backoff, streaming via SSE, cursor-based pagination, and error classification. The agent library wraps the Responses API, manages conversation state, and synthesizes structured events from raw stream data. The CLI uses `click` for command groups covering models, vector stores, eval, datasets, providers, and inference. Key data flows: users instantiate a client, call resource methods (e.g., `client.chat.completions.create()`), which serialize parameters via Pydantic models, send HTTP requests, and parse responses. Streaming responses are handled by `Stream`/`AsyncStream` classes that decode SSE events. The agent library consumes stream events and emits turn/step lifecycle events for UI or logging. External dependencies are minimal: `httpx`, `pydantic`, `typing-extensions`, and optional `aiohttp` for async transport. The codebase is well-structured but heavily generated; the custom agent code is the most reusable and demonstrates non-obvious skill in event synthesis and tool orchestration.
Wavefront is an enterprise AI middleware platform, open-source alternative to tools like UnifyApps and n8n. It provides a full-stack system for building, managing, and deploying AI agents, chatbots, voice agents, knowledge bases (RAG), and AI-powered workflows. The monorepo includes a Python FastAPI backend with a modular, domain-driven architecture (agents, tools, knowledge base, voice, triggers, auth, API proxy), a React/TypeScript management console, background workers (Celery, stream processors), and the `flo_ai` Python agent framework (LLM wrappers, Arium graph engine, tool system). The platform supports multi-tenancy, RBAC, cloud storage abstractions (GCP/AWS/Azure), and pluggable datasource and authenticator modules. While primarily an application, it incorporates RAG pipelines, image embedding services, and model inference, giving it a hybrid character with strong data engineering elements.
OpenCrew is a multi-agent operating system designed to extend OpenClaw into a manageable AI team. It enables users to create specialized agents (Chief of Staff, CTO, Builder, CIO, Knowledge Officer, Operations, Research) that collaborate through messaging platforms like Slack, Discord, and Feishu. The system implements structured collaboration through role-based channels, thread-based task isolation, and agent-to-agent communication protocols. Key features include autonomy levels (L0-L3), task classification (QAPS), two-mode agent collaboration (delegation and discussion), and three-layer knowledge distillation. The application is actively maintained with recent updates focusing on cross-agent collaboration capabilities.
This repository is a curated collection of 93+ production-ready AI engineering projects, spanning beginner to advanced levels. It serves as a practical resource for building AI applications, with a strong emphasis on agent orchestration (CrewAI, AutoGen, OpenAI Swarm), Retrieval-Augmented Generation (RAG) pipelines, Model Context Protocol (MCP) servers, and LLM evaluation frameworks. The dominant character is application development: most projects are deployable apps using Streamlit, Next.js, FastAPI, or as standalone MCP servers. Notable sub-projects include a full visual workflow builder (open-agent-builder) with Convex backend, a multi-modal RAG pipeline (notebook-lm-clone), and a comprehensive audio analysis toolkit. While primarily application-focused, the repository also contains data/research elements such as model fine-tuning scripts (GRPO, DPO, SFT), synthetic data generation (SDV), and LLM comparison dashboards with Opik/DeepEval evaluation. The codebase is heterogeneous in quality: some projects exhibit clean architecture and reusable patterns, while others are simple demos with boilerplate code and hardcoded dependencies. Overall, it is an excellent educational resource for AI engineers seeking practical, runnable examples, but not a single cohesive product.
This repository is a large-scale educational resource and community showcase for building LLM-powered agents using the custom 'HelloAgents' framework. It is organized as a multi-chapter tutorial (16 chapters) covering agent fundamentals, tool use, memory systems, multi-agent protocols (MCP/A2A/ANP), context engineering, reinforcement learning (SFT→GRPO), and evaluation. Each chapter contains runnable code examples, and the repository also hosts 40+ community-contributed 'Co-creation-projects' that demonstrate real-world agent applications across domains including stock analysis, health record management, academic research, code assistance, game design, and data analysis. The architecture is not a single deployable system but a polyglot collection of independent projects, most following a FastAPI backend + Vue/React frontend pattern, with a shared dependency on the HelloAgents framework core. The primary data flow in each project is: user input → LLM reasoning loop (ReAct/Plan-and-Solve/Reflection) → tool execution (search, database, code execution, APIs) → memory/context update → response generation. External dependencies include OpenAI-compatible LLM APIs, search engines (Tavily, Brave, DuckDuckGo), vector stores (Qdrant, Milvus, Neo4j), and various domain-specific APIs (financial data, academic databases, weather). The codebase serves as both a learning path for agent development and a practical reference for common agent patterns, though quality varies significantly between the polished tutorial chapters and the prototype-level community projects.
This repository provides a comprehensive CLI toolkit and programmatic interface (Python SDK and notebook clients) for creating, deploying, managing, monitoring, and evaluating AI agents on Amazon Bedrock AgentCore. The system supports multiple agent frameworks (Strands, LangChain/LangGraph, CrewAI, AutoGen, OpenAI Agents, Google ADK) and infrastructure-as-code options (CDK, Terraform). Key capabilities include project scaffolding via Jinja2 templates, runtime lifecycle management (configure, launch, destroy, invoke), memory management (short-term/long-term, semantic, user preference), observability (CloudWatch-based trace and log querying with OpenTelemetry span parsing), evaluation (LLM-as-a-judge with built-in and custom evaluators), policy engine CRUD, gateway management (MCP, API key, Cognito OAuth), and identity federation setup. The codebase is well-structured with a clear three-layer architecture (CLI -> operations -> services), extensive test coverage, and consistent error handling. However, the project is explicitly deprecated in favor of the @aws/agentcore NPM CLI, meaning future development is minimal and users are encouraged to migrate. Despite the deprecation, the reusable abstractions (e.g., CloudWatch query builder, trace processor, strategy models, configuration schemas) demonstrate solid software craftsmanship.
kimi-cli is MoonshotAI's Python AI coding assistant CLI, now explicitly archived and replaced by the TypeScript Kimi Code CLI. The pyproject.toml declares Development Status :: 7 - Inactive, the README states no further releases or security fixes will ship, and the final release (1.52.0) is a deprecation shim: every entry point short-circuits to a notice that downloads and installs the successor via a CDN script. Tests in the repo actively enforce that the legacy Typer CLI is unreachable. In its working life it was a full-featured terminal agent with interactive shell (prompt_toolkit/Rich), print mode, file editing tools, shell execution, web fetch/search, subagents, plan mode, approval workflows, background tasks, hooks, plugins, MCP server management, OAuth, and session persistence.
Vizra ADK is a retired Laravel package (final release v0.0.48) that provides a comprehensive framework for building AI agents with LLM orchestration, tool systems, memory, workflows, evaluations, tracing, and MCP integration. It is not a standalone application but a library designed to be installed into Laravel projects. The architecture follows Laravel package conventions: service providers, facades (Agent, Workflow), Eloquent models, Artisan commands, Livewire components, and database migrations. Key entry points include the Agent facade for running agents, the Workflow facade for orchestrating multi-agent workflows, and a set of CLI generators for scaffolding agents, tools, and evaluations. The system supports multiple LLM providers via Prism PHP, persistent memory (session, user, vector), tool chaining, sub-agent delegation, and an evaluation framework with LLM-as-a-Judge assertions. The codebase is well-structured with strong test coverage (Pest + Mockery), but the package is officially retired with no further development, leading to high technical drift as dependencies age out. Despite retirement, many components (e.g., ToolChain, VersionablePrompts, AgentVectorProxy, HasLogging) are self-contained and demonstrate reusable patterns suitable for extraction into new projects.
VOLTTRON is a Python-based distributed control and integration platform for building and industrial control systems. It runs cooperating 'agents' (services) that communicate over a ZeroMQ/RabbitMQ message bus called VIP, supporting pub/sub, RPC, and peer-to-peer routing. The platform provides agent lifecycle management (volttron-ctl), a web UI, authentication, historians, and protocol drivers for Modbus, BACnet, and similar field protocols. In production it acts as a central hub for collecting sensor/controller data, executing actuation commands, and hosting automation logic.
This repository is the archived reference implementation of the CAI (Cybersecurity AI) framework by Alias Robotics. It is a comprehensive, open-source platform for building and deploying AI-powered offensive and defensive security automation. The codebase provides a full-stack agent framework including a Python SDK (Agent, Runner, tools, handoffs, guardrails, tracing, voice, MCP), a CLI/REPL with rich interactive features, a Textual-based TUI, a FastAPI REST API server, and a benchmarking suite (CAIBench) with cyber ranges and CTF challenges. The framework supports multiple LLM providers (OpenAI, LiteLLM, custom) and includes specialized agents for red teaming, blue teaming, bug bounty, DFIR, network analysis, and more. It also features continuous operations (24/7 autonomous loops), session management, cost tracking, and extensive refusal testing (400+ scenarios). While the repository is now archived and will not receive further updates, the SDK components (function_tool decorator, Handoff abstraction, tracing processors, voice pipeline) are well-crafted and could be extracted for reuse in other agent-based projects. The codebase is large and monolithic, with some global mutable state and test duplication, but overall demonstrates strong engineering practices in the cybersecurity AI domain.
AgentKit is a full-stack framework for building constrained, reliable LLM agents, developed by BCG X and now deprecated. The codebase consists of a FastAPI backend (Python 3.10) that orchestrates LangChain-based agents through a meta-agent and a simple router, executing tools like SQL queries, PDF question-answering, visualizations, image generation, and summarization. The Next.js 14 frontend provides a conversational chat interface with streaming responses, tool action rendering, and configurable UI components. The application is Dockerized with PostgreSQL, Redis, Minio, and Caddy, and supports optional features such as user authentication (NextAuth), feedback collection (LangSmith), and Celery-based background tasks. The architecture is monolithic in code but service-oriented in deployment, with configuration-driven agent behavior via YAML files. While the codebase demonstrates solid engineering patterns (e.g., streaming callbacks, base tool classes, state management), it is explicitly deprecated, relies on outdated LangChain imports and API versions, and requires significant maintenance to be viable for new projects. The repository serves as a reference for building constrained agents but is not recommended for production use without substantial updates.