Beyond the Single-Best Model: Rashomon Partial Dependence Profile for Trustworthy Explanations in AutoML

TL;DR

This paper introduces the Rashomon Partial Dependence Profile, a new explanation method that captures uncertainty by aggregating multiple near-optimal models' feature effects, enhancing trustworthiness in AutoML.

Contribution

It proposes a novel framework that incorporates model multiplicity into explanations, addressing explanation uncertainty in AutoML systems.

Findings

Rashomon PDP covers less than 70% of the best model's PDP in most cases.

The method highlights areas of disagreement among models, revealing interpretive variability.

Rashomon PDP improves the reliability and trustworthiness of model explanations.

Abstract

Automated machine learning systems efficiently streamline model selection but often focus on a single best-performing model, overlooking explanation uncertainty, an essential concern in human centered explainable AI. To address this, we propose a novel framework that incorporates model multiplicity into explanation generation by aggregating partial dependence profiles (PDP) from a set of near optimal models, known as the Rashomon set. The resulting Rashomon PDP captures interpretive variability and highlights areas of disagreement, providing users with a richer, uncertainty aware view of feature effects. To evaluate its usefulness, we introduce two quantitative metrics, the coverage rate and the mean width of confidence intervals, to evaluate the consistency between the standard PDP and the proposed Rashomon PDP. Experiments on 35 regression datasets from the OpenML CTR23 benchmark suite show that in most cases, the Rashomon PDP covers less than 70% of the best model's PDP, underscoring the limitations of single model explanations. Our findings suggest that Rashomon PDP improves the reliability and trustworthiness of model interpretations by adding additional information that would otherwise be neglected. This is particularly useful in high stakes domains where transparency and confidence are critical.