$ show facebook/starlark-rust
This repository is a production-grade Rust implementation of the Starlark language, the deterministic Python-like configuration language used by build systems such as Bazel and Buck2. The system is structured as a multi-crate workspace: `starlark` provides the core runtime (parser, bytecode compiler, evaluator, garbage-collected heap, type system, and standard library), `starlark_syntax` handles parsing and AST, `starlark_map` offers high-performance ordered hash maps and sets, `starlark_derive` supplies procedural macros for defining Starlark-compatible Rust types, and `starlark_bin` bundles a CLI with REPL, linter, and Debug Adapter Protocol (DAP) server. Additional crates include `starlark_lsp` for Language Server Protocol support and a WASM example for embedding in JavaScript. The architecture is a classic interpreter pipeline: source text is parsed into an AST (`AstModule`), compiled to bytecode (with optimizations like constant folding, speculative execution, and inlining), and executed by a stack-based bytecode VM. Values are managed on a two-tier heap (unfrozen mutable and frozen immutable) with a custom bump allocator and explicit freeze operations. The type system supports both runtime type checking and static analysis, including union types, callable signatures, and generic type matchers. External dependencies include the `tower-lsp` and `lsp-types` crates for the LSP server, and several internal Facebook crates (e.g., `buck2_core`, `buck2_data`) for pagable serialization and profiling. Key entry points are `starlark::eval::eval_module` for programmatic evaluation, `starlark_bin/bin/main.rs` for the CLI, and the `#[starlark_module]` and `#[starlark_value]` proc macros for extending the language with custom types and functions. The system is designed for embedding in build tools, with a focus on performance, determinism, and debuggability.
The LLM identified 3 files worth highlighting for reuse:
Implements `SmallMap`, a memory-efficient ordered map that uses a small vector for up to a few entries and switches to a hash table for larger sizes. It guarantees deterministic iteration order (insertion order) and provides standard map operations. This component is a standalone crate (`starlark_map`) with no internal dependencies beyond standard Rust, making it directly reusable in any Rust project needing a compact, deterministic map.
Provides `Fx64Hasher`, a 64-bit variant of the Fx hash function with deterministic output across platforms and Rust versions. Includes golden test files to ensure hash stability, which is critical for serialization and cross-process consistency. This hasher is used throughout the Starlark runtime for maps, sets, and type IDs. It can be extracted as a standalone hasher for any application requiring reproducible hash values.
Implements `Alloca`, a lightweight bump allocator for temporary arrays that falls back to heap allocation when the stack-allocated buffer is exhausted. It is used internally for constructing bytecode and intermediate data structures. The pattern is self-contained and could be extracted as a reusable utility for any Rust project needing fast, short-lived allocations without global state.