$ show codestable/CodeStable
CodeStable is a framework for AI coding assistants (Claude Code, Codex CLI) that provides a set of 'skills' — thin harnesses that guide AI agents through software development tasks such as feature implementation, bug fixing, refactoring, epic decomposition, code review, and lesson management. The system emphasizes risk-based assurance, evidence-driven decisions, and minimal project memory, with a core philosophy of 'thin harness, thick context'. It is distributed as plugins for both Claude Code and Codex CLI, and includes a meta-skill (eval-cs-skill) for authoring, evaluating, optimizing, and releasing skills. The repository contains extensive experiment infrastructure with 15+ structured experiments covering routing accuracy, end-to-end outcomes, planted-defect recall, and learning transfer across multiple models. It features a sophisticated evaluation pipeline with deterministic oracles, hidden tests, regression tests, and checkpointed resume. The codebase demonstrates high craftsmanship with clean abstractions, strict validation, and cognitive honesty tagging (measured/soft/underpowered). Key architectural patterns include the thin skill pattern (compact SKILL.md with progressive disclosure via references), adapter pattern for AI model execution (Claude Code, Codex CLI, API, mock), deterministic seed building for reproducible experiments, and a risk-to-assurance mapping used generically by all task skills. The project has 8 ADRs documenting design decisions and a v1-to-v2 migration path that reduced 32 skills to 8. While the codebase is well-maintained and actively developed, some experiments are marked underpowered and the learning transfer epic was cancelled without completing primary hypotheses. The evaluation system is complex and tightly coupled to specific model names and macOS sandbox profiles, which may limit portability.
The LLM identified 5 files worth highlighting for reuse:
A standalone Python script that constructs a deterministic seed repository with staged git commits for reproducible experiments. It creates a task API project across multiple stages (core, projects, workflow, scheduling, HTTP API, onboard) with fixed commit dates and content hashes. This pattern is reusable for any project needing reproducible seed repos for benchmarking or testing AI coding assistants.
Abstract base class for harness adapters that execute AI models in isolated environments. Defines a common interface for running prompts with seatbelt profiles, cost estimation, and result collection. Concrete implementations exist for Claude Code (adapter_claude.py), Codex CLI (adapter_codex.py), direct API calls (adapter_api.py), and mock harnesses. The pattern is reusable for any system needing to run AI models in sandboxed, auditable environments.
Strict configuration loading and validation module for experiments. Defines a schema for experiment configs with required fields (id, kind, model, variants, fixtures) and enforces constraints like fixture ID uniqueness and path existence. The pattern is reusable for any project needing validated, schema-enforced configuration files.
Fixture loading module that validates and loads test fixtures from JSON files. Supports multiple answer types (findings-recall, routing-decision, e2e-outcome, learning-transfer) with strict schema validation per type. Includes fixture ID uniqueness checks and path resolution. This pattern is reusable for any testing or evaluation system that needs structured, validated test data.
A standalone validation script that checks the integrity of the plugin package. Verifies manifest fields, version consistency, skill layout, documentation commands, and interface contracts. It enforces that all required files exist and that version numbers match across manifests. This pattern is reusable for any project distributing plugins or packages that need automated integrity checks.