$ show VKCOM/nocc
nocc is a distributed C++ compiler system designed to accelerate C++ compilation, particularly for large projects with frequent rebuilds. It consists of three main components: a lightweight C++ wrapper (`nocc`) that pipes compiler commands to a Go daemon (`nocc-daemon`), which in turn coordinates with Go-based compilation servers (`nocc-server`). The system uses remote source and object caching to avoid re-uploading unchanged files and recompiling unchanged code, features an own includes parser for faster dependency detection than invoking the C++ preprocessor, supports precompiled headers compiled remotely, and provides local fallback when remote servers are unavailable. The architecture maintains persistent gRPC connections between daemons and servers while client processes start and die per compilation job, making it well-suited for build systems like CMake or KPHP that launch many parallel compilation processes.
The LLM identified 5 files worth highlighting for reuse:
Implements a custom SHA256 struct that breaks the hash into four uint64 fields for efficient XOR operations. This optimization allows quick comparison of file hashes without full byte-by-byte comparison, which is frequently used throughout the codebase for cache lookups. The component is completely self-contained with no external dependencies beyond the standard library and could be extracted for use in any project requiring efficient hash-based file identification.
Implements the .nocc-pch file format as a solution to the problem that traditional precompiled headers (.gch/.pch) don't work reliably across different machines. Instead of generating machine-specific pch files locally, nocc creates a text file listing all dependencies, which is uploaded to servers and compiled into real pch files remotely. This approach solves the cross-machine pch compatibility issue while maintaining the performance benefits of precompiled headers. The component is self-contained and demonstrates non-obvious skill in solving a complex distributed systems problem.
Implements an LRU (Least Recently Used) file cache that uses hard links for instant cache restoration. When a file is 'saved' to the cache, it creates a hard link to the original file, allowing near-instant retrieval without copying. The cache automatically evicts least recently used files when size limits are exceeded. This technique is used for both source and object caches in the system and represents a clever optimization for build acceleration scenarios where files are frequently reused but storage space is limited.
Handles parsing and generation of dependency files (.o.d files) for build systems that use -MD/-MF etc. flags. The component correctly implements the complex Makefile dependency file format, including handling of multi-line dependencies, escaping special characters, and supporting various flag combinations (-MD, -MMD, -MP, -MT, -MQ). It's well-encapsulated, depends mainly on standard library, and solves a recurring problem in build systems that could be reused in other projects needing dependency tracking.
Caches information about system headers to avoid unnecessary uploads. When a client reports a system header like /usr/include/math.h, the server checks if it matches its own copy (by size and SHA256) and if so, tells the client not to upload it since it can use the local copy. This optimization reduces network traffic and improves performance, especially for system headers that are commonly used but rarely change. The component is self-contained, thread-safe, and demonstrates a practical optimization for distributed compilation systems.