$ show sentient-agi/EvoSkill
EvoSkill is a CLI tool and Python library for automatically discovering and improving skills for coding agents. It implements a self-improvement loop: a base agent is evaluated on a task, failures are analyzed by a proposer agent which suggests skill/prompt mutations, a generator creates new skills/prompts, and the evolved agent is re-evaluated. The best programs are tracked via git branches, enabling frontier management and rollback. The system supports multiple agent SDKs (Claude Code, Codex CLI, OpenCode, OpenHands, Goose, Harbor) through a unified harness abstraction, allowing the same evolution loop to work across different agent platforms. The architecture is modular: CLI commands (init, run, eval, skills, diff, logs, reset, remote) provide the user interface; an API layer (EvoSkill, EvalRunner) offers programmatic access; the loop engine (SelfImprovingLoop) orchestrates the evolution; the registry (ProgramManager) handles git-based program versioning; the harness layer abstracts SDK-specific execution with retry, timeout, and structured output parsing; agent profiles define role-specific configurations (base agent, proposer, generator, evaluator); and evaluation modules support multiple scoring methods (exact match, tolerance, LLM-as-judge, script-based, Harbor). External dependencies include various agent SDKs (claude-agent-sdk, opencode-ai, openhands-ai, etc.), LLM providers (OpenRouter, Fireworks), and infrastructure tools (Docker, Daytona for remote execution). The repository also includes example benchmarks (OfficeQA, SEAL-QA, Dabstep, LiveCodeBench) with datasets and evaluation scripts, demonstrating the system's application to real-world tasks. The codebase exhibits strong engineering practices with clean abstractions, reusable components, and comprehensive testing, though there are some areas of duplication (e.g., skill-creator scripts) and global mutable state in the SDK configuration.
The LLM identified 5 files worth highlighting for reuse:
Provides the Agent class with exponential backoff retry (30s-60s-120s, 3 attempts, 20-minute timeout) and the AgentTrace Pydantic model for structured results. This abstraction decouples downstream code from SDK-specific execution details and can be reused to wrap any agent SDK with consistent retry and output parsing.
Implements RunCache that caches agent evaluation results keyed on a git tree hash of behavior-affecting files (skills, base prompt) combined with the question hash. This avoids redundant evaluations when program content hasn't changed, demonstrating a clever caching strategy for iterative improvement loops.
Implements FeedbackDescent, a generic open-ended text optimization algorithm using pairwise comparison. It operates as a proposer-evaluator loop with configurable early stopping, independent of domain. This can be extracted and applied to any text optimization problem where pairwise feedback is available.
Contains score_answer and fuzzy_match_answer functions that implement sophisticated numeric matching with unit detection (million/billion/trillion), text overlap on hybrid dates, multi-number list handling, and year filtering. This is a reusable evaluation component for QA tasks requiring tolerance-based scoring.
Implements ProgramManager that stores each program as a git branch (program/…) with .claude/program.yaml and skills, supporting frontier tracking via tags and rollback. This non-obvious use of git for program versioning provides a robust, auditable history of agent improvements.