Woodelf++: A Fast and Unified Partial Dependence Plot Algorithm for Decision Tree Ensembles

TL;DR

Woodelf++ is a fast, unified algorithm for computing various partial dependence and interaction metrics in decision tree ensembles, significantly outperforming existing methods in speed and efficiency.

Contribution

It introduces a unified framework for efficiently computing PDPs, Joint-PDPs, and PDIVs, with substantial complexity improvements and GPU support.

Findings

Computes PDPs and Joint-PDPs up to 6x faster than the state of the art.

Calculates All-Order-PDIVs in 5 minutes versus an estimated million years.

Supports GPU acceleration and exact Full PDPs for comprehensive model interpretation.

Abstract

Partial Dependence Plots (PDPs) visualize how changes in a single feature affect the average model prediction. They are widely used in practice to interpret decision tree ensembles and other machine learning models. Joint-PDPs extend this idea to pairs of features, revealing their combined effect. Partial Dependence Interaction Values (PDIVs) measure feature interactions. The Any-Order-PDIVs task computes these interactions for every feature subset across all rows of the dataset. We introduce Woodelf++, a unified and efficient approach for computing all these useful explainability tools on decision tree ensembles, building on Woodelf, an algorithm for efficient SHAP computation. By deriving suitable metrics over pseudo-Boolean functions, Woodelf++ can compute PDPs (exact and approximate), Joint-PDPs, and Any-Order-PDIVs in a unified framework. Our method delivers substantial complexity improvements over the state of the art, including an exponential gain for Any-Order-PDIVs. Additionally, we introduce and efficiently compute Full PDPs, which leverage the model's split thresholds to faithfully capture its behavior across all possible feature values. Woodelf++ is implemented in pure Python and supports GPU acceleration. On a dataset with 400,000 rows, Woodelf++ computes PDP and Joint-PDP up to 6x faster than the state of the art and up to five orders of magnitude faster than scikit-learn. For Any-Order-PDIVs, the gap is even larger: Woodelf++ computes all interaction values in 5 minutes, while the state of the art is estimated to require over 1,000,000 years.