$ show Evovest/EvoTrees.jl
EvoTrees.jl is a production-grade gradient-boosted decision tree (GBDT) library written entirely in Julia. It implements a histogram-based tree-growing algorithm with support for multiple loss functions (MSE, LogLoss, Poisson, Gamma, Tweedie, MAE, Quantile, MultiQuantile, credibility losses, Gaussian/Legistic MLE, and LambdaRank for ranking). The library provides both CPU (multi-threaded) and GPU (CUDA, ROCm, Metal via KernelAbstractions) backends, enabling efficient training and inference on diverse hardware. It is designed as a composable Julia package with a clean module structure, MLJ integration for seamless use within the Julia machine learning ecosystem, and serialization/deserialization via BSON for model persistence. At its core, the system implements a loss-type hierarchy (`src/loss.jl`) where each loss is an abstract type with inline scalar gradient/hessian functions that are shared across CPU and GPU code paths. The training algorithm binarizes features into 8-bit histograms, computes gradient statistics per bin, and finds optimal splits via gain scanning. Multi-target objectives (e.g., location/scale MLE) and monotonic constraints are supported. The library also includes SHAP value computation (Linear TreeShap), feature importance, tree plotting (Makie recipe), and a comprehensive callback system for evaluation and early stopping during training. While primarily a library for embedding in larger ML workflows, EvoTrees.jl also comes with an extensive collection of benchmarks, tutorials, and research experiments (credibility loss, multi-quantile, ranking). It is not a data-pipeline or analytics project but a well-architected software library that demonstrates advanced patterns in Julia numerics, GPU programming, and loss function abstraction. The repository is actively maintained, with regular commits, CI/CD, and comprehensive documentation built with Documenter.jl and Vitepress.
The LLM identified 4 files worth highlighting for reuse:
Defines an abstract type hierarchy for loss functions (GradientRegression, MLE2P, Cred, Quantile, etc.) with per-observation gradient and hessian functions implemented as @inline methods. Each loss type encapsulates its own gradient/hessian logic, link function, and initialisation. This design is self-contained, enables easy addition of new losses without altering the core algorithm, and is reused verbatim by both CPU and GPU code paths. It demonstrates a clean, extensible pattern for loss-function-oriented algorithms.
Implements the Linear TreeSHAP algorithm for computing SHAP values from a trained EvoTree ensemble. The code is a port of the algorithm by Yu et al. (2022) and is isolated in a single module with its own internal data structures (ShapTree, ShapEdge). It can be extracted and applied to any ensemble of binary decision trees, not just EvoTrees. The implementation is non-trivial and demonstrates expert-level handling of recursive tree traversal and linear-complexity SHAP computation.
Provides a dictionary-based metric registry with EvalMetric structs that encapsulate per-observation metric functions, aggregation, and a `is_maximise` trait for early-stopping direction. Includes support for grouped metrics (NDCG, Pearson) with thread-chunked computation. The design is reusable for any ML library needing extensible evaluation metrics, and the pattern of separating metric definition from calculation is clean and modular.
Implements a CallBack type that records evaluation metrics during training and supports early stopping when a metric stops improving (with patience and tolerance). The callback is called after each round and can be extended to log metrics or perform custom actions. This pattern is generic for any iterative training algorithm and demonstrates a clean separation of training logic from monitoring.