$ show OpenMined/PySyft
PySyft is a privacy-preserving data science platform that enables data owners to share private datasets and approve code execution without exposing raw data. The codebase is a monorepo with multiple interdependent packages: `syft` (sync engine, peer management, encryption), `syft-rds` (high-level remote data science client for datasets and jobs), `syft-job` (job lifecycle, runner, disclosures), `syft-datasets` (dataset storage with versioned protocol codecs), `syft-enclave` (confidential computing with TEE attestation via Confidential Spaces or Tinfoil, signed receipts), `syft-bg` (background services for auto-approval, email notifications, systemd management, TUI), `syft-permissions` (declarative ACL engine), `syft-restrict` (static code verification and obfuscation for private regions), `syft-migration` (generic versioned migration framework for objects), `syft-notebook-ui` (rich HTML representation for Jupyter), and `syft-perms` (file-level permission browser). The core workflow: data scientists submit Python jobs encapsulating analysis code → data owners approve (manually or automatically) → jobs run in sandboxed environments or TEE enclaves → results and logs are staged and released according to multi-party disclosure policies. The system uses Google Drive as the default sync transport with offline-first P2P file exchange, protocol versioning for backward compatibility, and cryptographic key pinning for trust.
The LLM identified 6 files worth highlighting for reuse:
MigrationRegistry and MigrationService implement a generic versioned schema evolution framework. Objects declare a canonical name and version; up/downgrade paths are registered as functions. Protocol schemas are frozen as JSON artifacts to detect drift. This pattern is isolated, depends only on Python stdlib and pydantic, and can be extracted for any application needing backward-compatible on-disk format evolution.
ACLTree implements a hierarchical policy engine that matches file paths to rulesets. Supports glob patterns, template variables ({{.UserEmail}}), and specificity sorting. Terminal rules stop traversal. The engine is self-contained with clear separation of compiled rules, matchers, and request resolution, making it reusable for any file-based permission system.
Evidence dispatch uses a runtime-detected provider pattern (detect/collect/describe) to wrap TEE attestation evidence from different backends (Confidential Space, Tinfoil). The base protocol is clean: a provider responds to a nonce-bound challenge with signed claims. The dispatch logic is isolated and could be reused to support new attestation providers with minimal changes.
This verifier performs static AST analysis to enforce a deny-by-default policy on Python code. It scans imports, function definitions, control flow, and operator usage against allowed/denied lists. It also detects bypass attempts like dynamic imports, attribute hijacking, and homoglyph attacks. The verifier is a standalone module (imports only stdlib AST and pydantic) that can be reused for any scenario requiring safe sandboxing of user-submitted Python code.
PersistedDict provides a dict-like interface backed by a JSON file with portalocker-based cross-process locking. It supports atomic rename, autosave, and and can be used as a simple local key-value store for caching or state across processes. The implementation is compact (under 200 lines) and isolated from the rest of the sync engine.
While this is a test fixture, the pattern of freezing release artifacts with specific protocol versions demonstrates a rigorous approach to backward compatibility testing. Each fixture contains a known-good state that can be round-tripped through current migration code. The methodology (commit fixture at release tag, tests compare against it) is reusable for any evolving on-disk format.