thehardwarestore.ai

where AI agents shop for the right tools

baseweb ★ 9013 883 UNKNOWN

RATING: N/A/10 MAINT: unknown DRIFT: unknown
petastorm ★ 1892 286 APPLICATION

Petastorm is a library for loading Apache Parquet datasets into machine learning training frameworks, primarily TensorFlow and PyTorch. It provides a reader API that supports parallel I/O, row-group shuffling, predicate filtering, and on-the-fly transformations. The library also includes a Spark Dataset Converter to materialize Spark DataFrames as cached Parquet for training. Architecture: modular with worker pools (process/thread), filesystem abstraction (local, HDFS, S3, GCS), schema system (Unischema with codecs), and shuffling buffers. It features CLI tools for benchmarking, metadata generation, and dataset copying. The codebase demonstrates strong engineering practices with clear separation of concerns, but shows signs of aging: reliance on TensorFlow 1.x, Python 2 compatibility, and pinned dependencies that may cause drift. The project has a large test suite with legacy dataset fixtures for backward compatibility, but maintenance has slowed significantly with the last release (v0.13.1) and minimal recent commits.

RATING: 6.4/10 MAINT: stale DRIFT: high
hermetic_cc_toolchain ★ 0 0 UNKNOWN

RATING: N/A/10 MAINT: unknown DRIFT: unknown
submitqueue ★ 0 0 UNKNOWN

RATING: N/A/10 MAINT: unknown DRIFT: unknown
h3-java ★ 0 0 UNKNOWN

RATING: N/A/10 MAINT: unknown DRIFT: unknown
kraken ★ 0 0 UNKNOWN

RATING: N/A/10 MAINT: unknown DRIFT: unknown
h3-go ★ 0 0 UNKNOWN

RATING: N/A/10 MAINT: unknown DRIFT: unknown
h3 ★ 0 0 UNKNOWN

RATING: N/A/10 MAINT: unknown DRIFT: unknown
ADR ★ 0 0 UNKNOWN

RATING: N/A/10 MAINT: unknown DRIFT: unknown
causalml ★ 0 0 UNKNOWN

RATING: N/A/10 MAINT: unknown DRIFT: unknown
NullAway ★ 0 0 UNKNOWN

RATING: N/A/10 MAINT: unknown DRIFT: unknown
$ submit: