An Empirical Evaluation of the Rashomon Effect in Explainable Machine Learning

TL;DR

This paper empirically investigates the Rashomon Effect in explainable machine learning, revealing how multiple models with similar performance can have diverse explanations, influenced by hyperparameters and metric choices.

Contribution

It offers a comprehensive empirical analysis of the Rashomon Effect's impact on explainability, highlighting the importance of hyperparameter tuning and metric selection.

Findings

Hyperparameter tuning influences explanation diversity

Metric choice significantly affects explanation comparability

Multiple models can produce similar performance but different explanations

Abstract

The Rashomon Effect describes the following phenomenon: for a given dataset there may exist many models with equally good performance but with different solution strategies. The Rashomon Effect has implications for Explainable Machine Learning, especially for the comparability of explanations. We provide a unified view on three different comparison scenarios and conduct a quantitative evaluation across different datasets, models, attribution methods, and metrics. We find that hyperparameter-tuning plays a role and that metric selection matters. Our results provide empirical support for previously anecdotal evidence and exhibit challenges for both scientists and practitioners.