Partio is a C++ library and toolkit for particle input/output and manipulation, primarily used in visual effects and simulation pipelines. It provides an abstraction layer for particle data (position, velocity, ID, etc.) with support for multiple file formats (BGEO, GEO, PDC, PDB, etc.) through a plugin-based I/O system. The repository includes core library code, command-line tools (partconvert, partinfo, partview), a Qt-based GUI editor (partedit.py), Python bindings via SWIG, and test suites. The architecture separates core data structures, format-specific handlers, and utilities like caching and KD-trees, enabling both direct library use and standalone tools for data inspection and conversion.
ScenarioNet is a data-centric toolkit for managing and converting autonomous driving datasets into a unified scenario format. It provides converters for Waymo, nuScenes, nuPlan, Argoverse 2, VoD, and procedural generation (PG) datasets, transforming raw data into a common ScenarioDescription structure compatible with the MetaDrive simulator. The repository includes dataset management operations (merge, split, filter, copy), verification tools (existence and simulation checks), and utilities for handling scenario metadata. While it contains training and evaluation scripts for machine learning models, its primary focus is on data preparation and scenario engineering for autonomous driving research. The data lineage involves ingesting raw datasets via dataset-specific toolkits, extracting scenario elements (object tracks, map features, traffic signals, timestamps), and serializing them into pickle-based dataset structures with mapping and summary files. These outputs enable downstream tasks such as multi-dataset aggregation, scenario filtering for ML training, simulation-based validation, and dataset splitting. The toolkit addresses research questions around dataset scalability, scenario diversity, and reproducible evaluation of autonomous driving systems by providing a standardized interface across heterogeneous data sources. Key reusable components include the modular converter utility for multiprocessing data transformation, generic dataset operations (merge/split/copy) that operate on the standardized scenario format, and a verification framework with parallel processing and error reporting. These components demonstrate solid data engineering practices but are tempered by dependency management issues (commented-out packages requiring manual installation), hardcoded paths in training scripts, and reliance on external SDKs that may introduce version drift.