$ show rogpeppe/go-internal
This repository is a curated collection of Go packages extracted from the Go standard library's internal packages, plus a few CLI tools built on top of them. The primary package is `testscript`, a robust script-based testing framework that uses txtar archives as test scripts. It provides a declarative way to write integration tests with built-in commands (exec, cmp, stdout, stderr, etc.) and supports external plugins via JSON-RPC. Other notable packages include `diff` (an O(n log n) anchored diff algorithm), `txtar` (a simple human-readable archive format), `lockedfile` (cross-platform atomic file operations with locking), `robustio` (transparent retry for ephemeral I/O errors), `cache` (build artifact cache with SHA256 indexing), and `par` (parallel work executor with memoization). The repo also includes CLI commands: `cmd/testscript` (standalone runner), `cmd/txtar-c` (archive creation), `cmd/txtar-x` (extraction), `cmd/txtar-addmod` (add module to archive), and `cmd/txtar-goproxy` (serve modules from archives). The architecture is modular; each package is independent and can be reused separately. There are thin forwarding layers to `golang.org/x/mod` and `github.com/google/renameio` for compatibility. The codebase demonstrates high craftsmanship in package design, platform-specific abstractions (build tags), and testing utilities. The primary domain is Go tooling and testing infrastructure.
The LLM identified 5 files worth highlighting for reuse:
A self-contained implementation of the Hunt-McIlroy diff algorithm with an anchored (patience) optimization. It computes the difference between two sequences of strings and returns line-by-line edits (equal, insert, delete). The API is simple: `Diff([]string, []string) []Change`. The algorithm is O(N log N) and handles duplicate lines correctly. It has zero external dependencies and can be extracted into any Go project that needs file or sequence comparison.
Implements the txtar archive format, a human-readable file format for bundling multiple files into a single text file. The format uses separator lines (`-- filename --`) and special quoting for filenames with special characters. The package provides `Parse` (from bytes), `Format` (to bytes), and helper functions for writing and quoting. It is widely used in Go's test ecosystem (e.g., testdata archives). This implementation is self-contained, stable, and can be reused in any project needing a simple, version-controllable archive format.
Provides a `File` type that behaves like `os.File` but uses a write lock to ensure exclusive access across processes. It handles platform-specific locking (fcntl on Unix, LockFile on Windows, with a fallback for Plan 9). The package additionally provides `Read`, `Write`, `Edit` functions for atomic file operations, including a mutex for in-process synchronization. It is designed to be a drop-in replacement for standard file I/O when mutual exclusion is needed. The implementation is production-ready and used in Go's own internal lockfile.
Defines helper functions for retrying file operations that fail due to ephemeral errors (e.g., `RemoveAll`, `Rename`). It uses a simple retry loop with a fixed delay (2s) and maximum attempts (10). This is particularly important on Windows, where file deletion often fails due to antivirus or open handles. The code is small, focused, and can be reused in any Go project that needs robust file manipulation, especially cross-platform. It also includes platform-specific variants for Darwin (flaky I/O) and Windows (retry delay).
Implements a parallel work executor (`Work` type) that runs a set of tasks in parallel with a configurable concurrency limit, and a `Cache` type for memoization of computation results. The `Cache` is thread-safe, supporting `Do` (compute once and cache) and `Get` (retrieve or compute). Both types use a stripe-locking scheme to reduce contention. This is a clean, reusable concurrency utility that can be extracted into any Go project needing parallel batch processing or memoization without external dependencies.