Miller (mlr) is a command-line data processing tool for structured text formats including CSV, TSV, JSON, JSON Lines, YAML, DKVP, and others. It operates as a streaming pipeline: input records are read from files or stdin, processed through a chain of transformer verbs (e.g., cut, sort, join, stats1, put, filter), and written to output in any supported format. The tool includes a domain-specific language (DSL) for record-level transformations, with support for user-defined functions, subroutines, loops, and higher-order functions. Miller is designed for performance, using Go channels for concurrent streaming, batch processing, and a slab allocator for efficient memory use. The architecture is a monolithic CLI application written in Go with no external runtime dependencies. Key entry points include the main binary (cmd/mlr/main.go), the entrypoint dispatcher (pkg/entrypoint), and the terminal subcommands (help, repl, script, regtest, mcp). The codebase is organized into packages for input readers, output writers, transformers (verbs), DSL parsing and execution, CLI flag handling, and a core value type system (Mlrval). Miller also provides an MCP (Model Context Protocol) server for AI agent integration, shell completion scripts, and a REPL for interactive use. Miller is not a research project; it is a production-grade data munging tool comparable to awk, sed, cut, join, and sort but for name-indexed data. Its primary data flows involve reading heterogeneous records, applying transformations via verbs or DSL expressions, and writing results. The tool handles compressed files, in-place editing, and output redirection to files or pipes.
VisiData is a terminal-based interactive spreadsheet application for exploring, cleaning, restructuring, and analyzing tabular data. It provides a real-time curses interface for loading data from diverse sources (CSV, JSON, Excel, SQL databases, APIs, etc.), applying transformations via hundreds of keyboard-driven commands (sort, filter, pivot, melt, unfurl, etc.), and saving results. The architecture centers around a Sheet/Column abstraction with a plugin system for loaders, features, and specialized applications (vgit for Git, vdsql for SQL). Key strengths include its extensible command system, dynamic type inference, and reversible operations via undo/redo. While the core is cohesive, the extensive optional dependencies and experimental features introduce some maintenance complexity.
YouPlot is a command-line tool for creating plots in the terminal using Unicode characters. It reads delimited data from stdin or files, parses it into headers and series, and generates various plot types (barplot, histogram, lineplot, scatter, density, boxplot, count) using the UnicodePlot backend. The tool follows Unix conventions, designed for use in data pipelines with options to pass through input data and output plots to stderr/stdout. It features progressive mode for real-time data visualization and supports configuration via CLI flags, config files, and sensible defaults.
WinMerge is an open-source differencing and merging tool for Windows, written in C++ with MFC and ATL. It provides file and folder comparison (2- and 3-way), syntax highlighting, inline editing, patch generation, and archive support. The application is structured as a monolith with a plugin system (COM-based DLLs) for pre-diff filtering, unpacking, and transformation. It integrates tightly with Windows Explorer via a shell extension and supports internationalization, dark mode, and extensive customization through property pages. The architecture follows the MFC document/view pattern, with core diff engines (vendored xdiff and GNU diffutils), a filter engine with regex-based line/folder filtering, and an options manager using registry or INI file backends. Third-party dependencies are heavily vendored, including Boost (MPL, Fusion, Preprocessor), Poco (Foundation, XML, Encodings), tree-sitter, CrystalEdit, and libxdiff. The project is actively maintained with regular releases, a comprehensive manual, and automated CI/CD pipelines. While primarily a Windows desktop application, it showcases reusable patterns such as an extensible options management system, a self-contained byte comparison engine, a lightweight async task wrapper, and a property page host. However, the codebase carries significant legacy MFC idioms and platform-specific code, limiting cross-platform portability. The vendored dependency tree is large, and some preprocessor-heavy Boost headers are fragile with compiler updates.