$ show rtk-ai/rtk
rtk is a Rust CLI proxy and hook system designed to minimize LLM token consumption for AI coding agents by filtering and compressing the output of shell commands. It operates as a command wrapper that intercepts commands like git status, pytest, or cargo test before they reach an agent's context window, runs the real tool, and emits a compact, token-optimized summary while preserving diagnostic value. The tool includes a sophisticated architecture: a core runner with a 'never-worse' guard that ensures filtered output never exceeds raw output in size, tiered parsing (JSON → regex → passthrough) for dozens of ecosystems, a declarative TOML filter DSL compiled into the binary, and a SQLite-based recall store for content-addressed output retrieval. It also features a permission model for hooks that can allow, deny, or ask on command rewrites, and integrates with 18+ AI agents (Claude Code, Copilot, Cursor, etc.) via custom hook formats. The repo is a single Rust binary with subcommands for execution, analytics (rtk gain), and session discovery (rtk discover), supported by an extensive test suite with fixture-based golden tests. While not a data research project, it incorporates data-driven elements like token savings accounting and telemetry, but its dominant character is a high-craft CLI application.
The LLM identified 4 files worth highlighting for reuse:
Generic streaming process execution framework with pluggable StreamFilter, BlockHandler, and LineHandler traits. Handles stdin/stdout/stderr filtering, tolerates non-UTF-8 output, caps raw output at 10 MiB, and forwards Unix signals (SIGINT/SIGTERM) to child processes. This is a self-contained pattern for any CLI tool that needs to run subprocesses and transform their output in real time, and could be extracted into a standalone library with minimal changes.
Dialect-aware CLI argument tokenizer (POSIX vs MSBuild) that correctly handles value consumption, attached short flags, '--' boundaries, and subcommand detection via configurable grammar tables. Provides helpers like has_flag, double_dash_flag_value, and injection_point used across multiple command filters. This solves a recurring problem in CLI tooling—accurately splitting command lines without shell execution—and is highly reusable in any project that needs to parse or rewrite command arguments.
Content-addressed recall store backed by SQLite with gzip compression, configurable retention/eviction, and env-var overrides. Stores command output keyed by a truncated SHA hash, supporting strict byte-size tracking, deduplication, and disabled-store behavior. This is a well-isolated data persistence pattern useful for any application that needs to cache, compress, and retrieve large process outputs with retention policies.
Defines the OutputParser trait and ParseResult enum with Full, Degraded, and Passthrough states, plus canonical types (TestResult, LintResult) and format modes. This establishes a consistent, extensible pattern for parsing external tool output, with graceful degradation from structured JSON to regex fallback to raw passthrough. It is a reusable abstraction for any project that must parse varied, version-sensitive command output while preserving a 'never-worse' guarantee.