$ show shap/shap
SHAP (SHapley Additive exPlanations) is a mature Python library for interpreting machine learning model predictions using Shapley values from cooperative game theory. It provides a unified API across model types: tree ensembles (XGBoost, LightGBM, CatBoost, sklearn, PySpark), deep neural networks (TensorFlow, PyTorch), linear models, and arbitrary callables. The architecture is modular: explainers (Tree, Deep, Kernel, Partition, Gradient, Linear, etc.) consume models and maskers (tabular, image, text) to produce Explanation objects, which are visualized via matplotlib or interactive JavaScript force plots. The library includes benchmarks for evaluating explanation quality, extensive test suites with serialization baselines, and C++/CUDA extensions for performance. Although primarily a library (application), it has a strong research character: it originated from academic papers, includes notebooks reproducing experiments, and contains a user study on human interpretability. The codebase is well-engineered with reusable patterns (serialization, OpChain for lazy operations, MaskedModel abstraction), but carries some technical drift from legacy API aliases, dependency version sprawl, and a large minified JavaScript bundle in the repository.
The LLM identified 5 files worth highlighting for reuse:
Provides a Serializable base class with versioned save/load using a flexible Serializer/Deserializer framework supporting multiple backends (pickle, cloudpickle, numpy, custom). All major objects (Model, Masker, Explainer) inherit from this, enabling robust persistence of complex state. Reusable in any project needing versioned serialization of hierarchical objects.
Implements the TreeExplainer and the TreeEnsemble/SingleTree parsers that convert models from XGBoost, LightGBM, CatBoost, sklearn, PySpark, and causal trees into a dense internal representation. The parsing logic is cleanly separated from the SHAP computation, making it reusable for other tree-based analysis tools that need to inspect internal structure across frameworks.
MaskedModel wraps a model, masker, and link function to efficiently evaluate predictions on masked subsets. It handles batching, automatic background data sampling, and incremental (delta) masking to avoid recomputation. This is a core abstraction used by many explainers and can be extracted for any perturbation-based attribution or feature importance analysis.
Defines the Masker base class with a compositional pattern (masking, unmasking, data generation) and subclasses for tabular (Independent, Partition), image, text, fixed, and composite masks. The interface is clean and could serve as a template for custom masking strategies in other interpretability or data augmentation frameworks.
A React/D3 component that renders an interactive force plot for Shapley value explanations. It is published as the standalone 'shapjs' npm package, demonstrating a pattern for packaging Python library visualizations as reusable web components. The component accepts data, colors, and link functions, making it adaptable to other additive explanation formats.