$ show toon-format/toon
TOON (Token-Oriented Object Notation) is a human-readable serialization format designed to minimize LLM token usage while preserving lossless round-trip conversion with JSON. This monorepo provides the reference TypeScript implementation (`@toon-format/toon`) and a CLI tool (`@toon-format/cli`) for encoding/decoding TOON. The library features streaming encode/decode, tabular forms for uniform data, normalization of special JavaScript values (BigInt, Date, Set, Map), security hardening against prototype pollution, and a replacer function with path tracking. The CLI supports stdin/stdout conversion, auto-detection by file extension, token statistics, and delimiter options. The repository also includes an extensive benchmark suite comparing TOON against JSON, YAML, XML, and CSV across token efficiency and retrieval accuracy using multiple LLMs (Claude, Gemini, GPT, Grok). Results show TOON matches JSON's accuracy while using ~43% fewer tokens. A documentation site with an interactive playground is provided via VitePress. The codebase is well-structured with clean abstractions, streaming APIs, and thorough testing, though it relies on an external spec repository and has a single author, creating bus factor risk.
The LLM identified 4 files worth highlighting for reuse:
Implements a streaming decoder that produces JsonStreamEvent tokens (startObject, endObject, key, primitive, etc.) from TOON input, enabling incremental processing without building a full AST. The pattern is self-contained and can be extracted for any streaming JSON-like format parser.
Streaming encoder that yields TOON lines from a JSON value, supporting tabular forms and delimiter options. The line-based output pattern is reusable for any line-oriented serialization format.
Handles special JavaScript values (BigInt, Date, Set, Map, -0, NaN, Infinity) for lossless round-trip encoding. Demonstrates careful edge-case handling in serialization and can be reused in any JSON-compatible encoder.
Provides setOwnProperty that uses Object.defineProperty to safely set properties including __proto__ as own data property, preventing prototype pollution. Reusable for any JSON parsing that needs to preserve __proto__ keys.