EvoSkill is a CLI tool and Python library for automatically discovering and improving skills for coding agents. It implements a self-improvement loop: a base agent is evaluated on a task, failures are analyzed by a proposer agent which suggests skill/prompt mutations, a generator creates new skills/prompts, and the evolved agent is re-evaluated. The best programs are tracked via git branches, enabling frontier management and rollback. The system supports multiple agent SDKs (Claude Code, Codex CLI, OpenCode, OpenHands, Goose, Harbor) through a unified harness abstraction, allowing the same evolution loop to work across different agent platforms. The architecture is modular: CLI commands (init, run, eval, skills, diff, logs, reset, remote) provide the user interface; an API layer (EvoSkill, EvalRunner) offers programmatic access; the loop engine (SelfImprovingLoop) orchestrates the evolution; the registry (ProgramManager) handles git-based program versioning; the harness layer abstracts SDK-specific execution with retry, timeout, and structured output parsing; agent profiles define role-specific configurations (base agent, proposer, generator, evaluator); and evaluation modules support multiple scoring methods (exact match, tolerance, LLM-as-judge, script-based, Harbor). External dependencies include various agent SDKs (claude-agent-sdk, opencode-ai, openhands-ai, etc.), LLM providers (OpenRouter, Fireworks), and infrastructure tools (Docker, Daytona for remote execution). The repository also includes example benchmarks (OfficeQA, SEAL-QA, Dabstep, LiveCodeBench) with datasets and evaluation scripts, demonstrating the system's application to real-world tasks. The codebase exhibits strong engineering practices with clean abstractions, reusable components, and comprehensive testing, though there are some areas of duplication (e.g., skill-creator scripts) and global mutable state in the SDK configuration.
ClawBench is a rigorous execution-first benchmarking framework for evaluating AI agents (particularly OpenClaw) on multi-step tasks. The system orchestrates agent runs through a gateway or containerized native evaluation, scores performance across four axes (completion, trajectory, behavior, advisory judge), and generates detailed diagnostic reports. Its architecture includes a job queue with HuggingFace Dataset persistence, a Gradio web app for interactive leaderboards, and a comprehensive CLI for running evaluations and analyzing results. The framework supports multiple agent runtimes via a pluggable adapter pattern (OpenClaw, Hermes, Claude Code, Codex) and provides a canonical task schema for agent-agnostic task definitions. Beyond basic scoring, ClawBench applies dynamical-systems and statistical methods to agent behavior: trajectory regime classification (trapped, limit-cycle, diffusive), Kaplan-Meier survival analysis, constraint index computation, and variance decomposition. It includes a configuration-space benchmarking system (v0.5) that fingerprints plugin profiles for cold-start prediction and fANOVA factor analysis. The codebase is well-tested with over 60 test files, uses Docker for reproducibility, and includes CI/CD pipelines for automated evaluation campaigns. However, the system is complex and has some hardcoded paths and environment variables tied to specific deployment contexts (Crabbox, OpenClaw gateway), and its heavy dependency on external services introduces maintenance risk. Overall, it is a production-grade benchmark with sophisticated analysis capabilities.
This repository is a comprehensive family of GUI automation agents (Mobile-Agent series) developed by Tongyi Lab, Alibaba Group. It provides end-to-end solutions for automating interactions across mobile (Android), desktop (Windows/macOS/Linux), and web platforms using vision-language models. The system perceives GUI elements through screenshots, grounds natural language instructions to actionable commands (click, type, scroll), and executes tasks via platform-specific controllers. The architecture includes multiple agent versions (v1-v3.5, E, UI-S1, PC-Agent) with shared core modules for perception, action parsing, and controller interfaces, alongside training frameworks (VERL) and evaluation suites (Android World, MiniWoB, Mind2Web). Key entry points are version-specific run scripts (e.g., run_ma35.py) and API services for remote agent control. The system relies on external dependencies like ADB for Android, PyAutoGUI for desktop, Playwright for web, and VLMs (Qwen series) for reasoning.