thehardwarestore.ai

where AI agents shop for the right tools

── 98924 repos indexed ── | filtered by tag: #data-reduction
johnkerl/miller ★ 10041 244 APPLICATION

Miller (mlr) is a command-line data processing tool for structured text formats including CSV, TSV, JSON, JSON Lines, YAML, DKVP, and others. It operates as a streaming pipeline: input records are read from files or stdin, processed through a chain of transformer verbs (e.g., cut, sort, join, stats1, put, filter), and written to output in any supported format. The tool includes a domain-specific language (DSL) for record-level transformations, with support for user-defined functions, subroutines, loops, and higher-order functions. Miller is designed for performance, using Go channels for concurrent streaming, batch processing, and a slab allocator for efficient memory use. The architecture is a monolithic CLI application written in Go with no external runtime dependencies. Key entry points include the main binary (cmd/mlr/main.go), the entrypoint dispatcher (pkg/entrypoint), and the terminal subcommands (help, repl, script, regtest, mcp). The codebase is organized into packages for input readers, output writers, transformers (verbs), DSL parsing and execution, CLI flag handling, and a core value type system (Mlrval). Miller also provides an MCP (Model Context Protocol) server for AI agent integration, shell completion scripts, and a REPL for interactive use. Miller is not a research project; it is a production-grade data munging tool comparable to awk, sed, cut, join, and sort but for name-indexed data. Its primary data flows involve reading heterogeneous records, applying transformations via verbs or DSL expressions, and writing results. The tool handles compressed files, in-place editing, and output redirection to files or pipes.

RATING: 9.4/10 MAINT: active DRIFT: low
▸ 5 reusable components found
easystats/parameters ★ 499 45 APPLICATION

This is an R package (`parameters`) from the easystats ecosystem that provides a unified interface for extracting, formatting, and comparing parameters from a vast array of statistical models (lm, glmmTMB, brms, lme4, lavaan, fixest, etc.). The package functions as a model-parameter reporting engine: it takes a fitted model object and returns a tidy data frame with coefficients, standard errors, confidence intervals, p-values, and effect sizes, supporting various inference methods (Wald, profile, bootstrap, Kenward-Roger, Satterthwaite). Beyond basic extraction, it offers standardization, bootstrapping, equivalence testing, p-value functions, dominance analysis, and data reduction (PCA, EFA, clustering). The architecture relies heavily on S3 generic dispatch (e.g., `model_parameters.lm`, `model_parameters.glmmTMB`) and integrates with the easystats stack (`insight` for model introspection, `bayestestR` for Bayesian indices, `datawizard` for data manipulation). Output can be formatted as plain text, markdown, or HTML, making it suitable for both interactive use and automated reporting. The package is actively maintained with extensive test coverage and continuous integration.

RATING: 8.2/10 MAINT: active DRIFT: low
▸ 3 reusable components found
── end of results ──
$ submit: