$ show hexo-ai/sia
SIA (Self-Improving AI) is a framework that orchestrates a three-agent loop (meta-agent, target agent, feedback/improvement agent) to iteratively improve an AI system's performance on a benchmark task. In production, it runs as a CLI tool (`sia run`) that sets up a virtual environment, loads task files, and executes generations: the meta-agent reads `task.md` and generates `target_agent.py`, the target agent executes the task and logs results, and the feedback agent reviews the execution and rewrites the target agent for the next generation. A FastAPI web dashboard auto-launches to visualize per-generation accuracy, agent code, trajectories, and improvement plans. The architecture is a CLI-driven monolith with a plugin system for agent backends (Claude, OpenHands, PydanticAI), JSON-based provider/profile configuration, and Docker sandboxing for secure target-agent execution. It relies on external LLM APIs (Anthropic, OpenAI, Gemini, Nebius, OpenRouter, Together) and optionally on Modal/SandboxFusion for the RL-based weights mode. The repository also bundles four benchmark tasks (GPQA, lawbench, longcot-chess, spaceship-titanic) with public/private data splits and evaluators, giving it a hybrid character that includes data/research components. However, its primary function is as an application for running and visualizing self-improvement experiments.
The LLM identified 4 files worth highlighting for reuse:
Implements a clean plugin registration system for agent backends. The `register` function allows modules to self-register, and `get_agent_impl` resolves by name with lazy imports. This pattern is isolated and can be extracted as a lightweight plugin framework for any Python project.
Provides a declarative configuration resolution system for LLM providers. It resolves provider definitions from JSON files by name or file path, with fallback to bundled defaults and user overrides. The pattern is generic and reusable for any project needing layered JSON configuration with sensible defaults.
Offers safe, size-limited file and JSON reading utilities with type validation. Functions like `read_json_safe` and `read_file_limited` prevent resource exhaustion and provide clear error messages. This is a set of universally useful I/O helpers that can be dropped into any Python project.
Defines `TargetAgentResult` and `FeedbackContext` dataclasses that form the stable internal wire contract between orchestrator components. The `.as_tuple()` method ensures backward compatibility with older output formats. This demonstrates a robust approach to versioned data transfer objects in evolving systems.