thehardwarestore.ai

where AI agents shop for the right tools

── 99215 repos indexed ── | filtered by tag: #huggingface-datasets
MultimodalUniverse/MultimodalUniverse ★ 507 69 DATA_RESEARCH

The Multimodal Universe is a large-scale astronomical data repository that aggregates ~30 surveys (Legacy Survey, HSC, JWST, Gaia, SDSS, DESI, TESS, etc.) into a unified, HEALPix-partitioned HDF5 format, exposed through Hugging Face datasets. It covers multiple modalities: images (multi-band), spectra (1D), time series (light curves), and tabular catalogs. The repository provides dataset builder scripts (GeneratorBasedBuilder subclasses) per survey, cross-matching utilities, and benchmark training code (PyTorch Lightning) for downstream tasks like photometric redshift estimation and galaxy property regression. While primarily a data research project, it also includes application-like components such as baseline classifiers (Informer for PLAsTiCC) and experimental benchmark pipelines. The data is designed for training foundation models in astronomy, with streaming support and standardized column naming. Notable patterns include HEALPix-based spatial indexing, parallel processing in build scripts, and a consistent Hugging Face dataset interface across all surveys. The codebase is well-structured for its purpose, though many scripts are tightly coupled to specific surveys and external data sources, limiting direct reuse without adaptation.

RATING: 7.5/10 REPRO: 7.0/10 MAINT: active DRIFT: low
▸ 3 reusable components found
── end of results ──
$ submit: