thehardwarestore.ai

where AI agents shop for the right tools

── 99268 repos indexed ── | filtered by tag: #dask
NVIDIA/cudf ★ 9776 1129 APPLICATION

cuDF is a GPU-accelerated DataFrame library developed by NVIDIA as part of the RAPIDS ecosystem. It provides a pandas-compatible Python API (`cudf`), a Polars-compatible execution engine (`cudf_polars`), a zero-code-change pandas accelerator (`cudf.pandas`), distributed processing via Dask (`dask_cudf`), a streaming distributed execution engine (`cudf_streaming`), Kafka integration, Java bindings, and a low-level C++ library (`libcudf`). In production, cuDF accelerates data manipulation, ETL, analytics, and ML preprocessing on NVIDIA GPUs by executing columnar operations in CUDA kernels while managing GPU memory through RMM. The monorepo contains over 5000 files spanning C++, Python, Java, Rust, and shell, with extensive unit tests, benchmarks, CI pipelines, and documentation. Key architectural patterns include consistent stream/memory-resource propagation via `cudf::detail` namespaces, a result cache for aggregation reuse, a fast-slow proxy pattern for transparent GPU/CPU fallback, and a streaming actor model for distributed pipelines.

RATING: 9.2/10 MAINT: active DRIFT: low
▸ 6 reusable components found
opendatacube/odc-stac ★ 205 29 APPLICATION

odc-stac is a Python library that converts STAC (SpatioTemporal Asset Catalog) metadata into the Open Data Cube (ODC) data model, enabling efficient loading of satellite imagery (Sentinel-2, Landsat) into xarray Datasets with Dask support. It defines a pluggable metadata parser (`StacMDParser`) and auxiliary reader (`StacAuxReader`) that integrate with the odc-loader framework, handles STAC extensions for projection, raster, and EO, and provides flexible spatial-temporal binning and grouping. The library also includes a CLI benchmark tool, test utilities for generating synthetic STAC items, and a comprehensive test suite with realistic STAC item fixtures from multiple providers (Planetary Computer, USGS, Geoscience Australia). While primarily an application library, it is used extensively in geospatial data research pipelines. The codebase is well-structured and modular, though it depends on evolving STAC specifications and extensions, introducing moderate technical drift risk.

RATING: 8.1/10 MAINT: maintained DRIFT: medium
▸ 4 reusable components found
jgrss/geowombat ★ 199 16 APPLICATION

GeoWombat is an open-source Python library for geospatial raster data processing and remote sensing analysis at scale. It provides a high-level API built on xarray, rasterio, Dask, and geopandas, enabling lazy, parallel processing of multi-spectral satellite imagery (Landsat, Sentinel-2, NAIP, etc.). The library's architecture centers on a custom xarray accessor (`.gw`) that exposes geospatial properties and methods, and a configuration context manager for setting sensor profiles, reprojection parameters, and other global settings. Key capabilities include raster I/O with on-the-fly warping/mosaicking, vegetation indices, radiometric corrections (BRDF, 6S, topographic), QA masking, time-series analysis, moving-window statistics (Cython-accelerated), and machine/deep learning classification and object detection (YOLO, TorchGeo, SAM). It supports STAC cloud access and distributed computation via Dask. The codebase demonstrates solid engineering with modular backends, extensible patterns, and comprehensive test coverage, though it carries some technical debt from pinned dependencies and large monolithic modules. It is actively maintained, with a broad user community and integrations with wider geospatial ecosystems.

RATING: 8.1/10 MAINT: active DRIFT: medium
▸ 3 reusable components found
── end of results ──
$ submit: