$ show inference-labs-inc/dsperse
dsperse is a distributed Zero-Knowledge Machine Learning (zkML) toolkit that decomposes ONNX neural network models into circuit-compatible segments for verifiable AI inference. The core Rust library provides model slicing, circuit compilation (using JSTprove backend), witness generation, and proof verification. It exposes both a CLI interface and Python bindings via PyO3/Maturin for integration into ML pipelines. The architecture follows a clear separation: the slicer module handles ONNX graph decomposition and tiling strategies (channel split, dimension split, spatial tiling), the pipeline orchestrates execution with memory-efficient tensor caching, and the backend trait enables pluggable ZK proving systems. Content-addressed storage with SHA256 hashing ensures artifact integrity across distributed execution. Notable constraints: the license explicitly restricts operational use without written permission from the IP owner, and the codebase relies on nightly Rust toolchain with specific git dependencies (tract-onnx, jstprove_circuits) that create version coupling risks. Security advisories are partially ignored in the audit configuration.
The LLM identified 5 files worth highlighting for reuse:
In-memory cache for intermediate activations during distributed inference. Decouples computation from storage, enabling memory-efficient execution across slices. The design allows tensor lifecycle management independent of the execution pipeline, making it reusable for any memory-constrained inference scenario.
Implements the Strategy pattern for execution modes (Single, Tiled, ChannelSplit, DimSplit). Each strategy handles different slice types uniformly through a common trait interface. This abstraction allows the pipeline to select execution modes based on slice metadata without coupling to specific implementation details.
ONNX graph optimization passes including constant folding, batch norm fusion, and dead node elimination. These transformations standardize models before circuit compilation, ensuring compatibility across different ONNX versions and backend requirements. The pattern of normalization before processing is reusable for any ONNX-based pipeline.
Implements content-addressed storage using SHA256 hashing for model components and weights. Generates manifest files for distribution and includes symlink rejection for path traversal protection. The pattern of hashing file contents for deduplication and integrity verification is applicable to any artifact distribution system.
Defines the ProofBackend trait that enables pluggable ZK proving systems. The trait abstracts proof generation, verification, and setup operations, allowing the pipeline to work with different backends without modification. This pattern is valuable for any system requiring backend interchangeability.