$ show Kaggle/kagglehub
kagglehub is the official Python library for interacting with Kaggle resources. It provides a unified API to download and upload models, datasets, competition data, and notebook outputs, with automatic handling of caching, authentication (including Colab secrets and JWT), and environment-specific resolution (direct HTTP, Kaggle notebook cache, Colab mount). The library uses a multi-resolver architecture where each resource type (model, dataset, etc.) is resolved by a chain of resolvers that are tried in priority order (Colab cache > Kaggle cache > HTTP). It also includes dataset loading adapters for pandas, polars, and Hugging Face Datasets, and supports Kaggle Packages (executable notebooks that export Python modules). The codebase is well-structured, extensively tested (with unit and integration tests using Flask stubs), and follows modern Python practices (type hints, linting, hatch build system).
The LLM identified 4 files worth highlighting for reuse:
Implements GCS resumable blob uploads with parallel file uploads (up to 8 threads), progress bars, retry logic, and automatic fallback to zipped archive when too many files (>50). Includes `filtered_walk` for walking directories with ignore patterns and `UploadDirectoryInfo` structure for organizing upload metadata. This module is self-contained (only depends on requests, tqdm, and kagglesdk types) and could be extracted to handle generic file uploads to cloud storage.
A small but critical module that normalizes and validates dataset file paths to prevent directory traversal attacks. It handles Windows/Unix path differences, rejects absolute paths, null bytes, and components like '..'. The `validate_dataset_path_descendant` function ensures server-returned paths stay within the requested directory. This is a reusable security pattern for any API that exposes file paths.
Implements a file-extension-based dispatch pattern for loading Kaggle datasets into pandas DataFrames. Validates file extension, selects the appropriate `pd.read_*` function, supports static kwargs (e.g., `sep='\t'` for TSV) and user-provided kwargs, and handles SQLite queries via a wrapped `read_sql_query`. The pattern is cleanly separated from download logic and could be reused for any file-based data loading with multiple backends.
Defines `PackageScope` for executing Kaggle Package imports within a context that provides versioned datasource resolution and asset path retrieval. Uses `contextvars` for thread-safe scope management. `_apply_context_manager_to_module` decorates all functions/methods in a module to automatically enter the scope. This pattern enables sandboxed, documented imports of third-party code and could be adapted for any dynamic module loading with context.