PySyft is a privacy-preserving data science platform that enables data owners to share private datasets and approve code execution without exposing raw data. The codebase is a monorepo with multiple interdependent packages: `syft` (sync engine, peer management, encryption), `syft-rds` (high-level remote data science client for datasets and jobs), `syft-job` (job lifecycle, runner, disclosures), `syft-datasets` (dataset storage with versioned protocol codecs), `syft-enclave` (confidential computing with TEE attestation via Confidential Spaces or Tinfoil, signed receipts), `syft-bg` (background services for auto-approval, email notifications, systemd management, TUI), `syft-permissions` (declarative ACL engine), `syft-restrict` (static code verification and obfuscation for private regions), `syft-migration` (generic versioned migration framework for objects), `syft-notebook-ui` (rich HTML representation for Jupyter), and `syft-perms` (file-level permission browser). The core workflow: data scientists submit Python jobs encapsulating analysis code → data owners approve (manually or automatically) → jobs run in sandboxed environments or TEE enclaves → results and logs are staged and released according to multi-party disclosure policies. The system uses Google Drive as the default sync transport with offline-first P2P file exchange, protocol versioning for backward compatibility, and cryptographic key pinning for trust.
ScenarioNet is a data-centric toolkit for managing and converting autonomous driving datasets into a unified scenario format. It provides converters for Waymo, nuScenes, nuPlan, Argoverse 2, VoD, and procedural generation (PG) datasets, transforming raw data into a common ScenarioDescription structure compatible with the MetaDrive simulator. The repository includes dataset management operations (merge, split, filter, copy), verification tools (existence and simulation checks), and utilities for handling scenario metadata. While it contains training and evaluation scripts for machine learning models, its primary focus is on data preparation and scenario engineering for autonomous driving research. The data lineage involves ingesting raw datasets via dataset-specific toolkits, extracting scenario elements (object tracks, map features, traffic signals, timestamps), and serializing them into pickle-based dataset structures with mapping and summary files. These outputs enable downstream tasks such as multi-dataset aggregation, scenario filtering for ML training, simulation-based validation, and dataset splitting. The toolkit addresses research questions around dataset scalability, scenario diversity, and reproducible evaluation of autonomous driving systems by providing a standardized interface across heterogeneous data sources. Key reusable components include the modular converter utility for multiprocessing data transformation, generic dataset operations (merge/split/copy) that operate on the standardized scenario format, and a verification framework with parallel processing and error reporting. These components demonstrate solid data engineering practices but are tempered by dependency management issues (commented-out packages requiring manual installation), hardcoded paths in training scripts, and reliance on external SDKs that may introduce version drift.