Global Aggregations of Local Explanations for Black Box models

TL;DR

This paper introduces GALE, a method for aggregating local explanations to better understand the global decision-making process of black box models, addressing limitations of existing approaches like LIME.

Contribution

The paper proposes GALE, a novel aggregation technique that improves the reliability of global insights derived from local explanations of black box models.

Findings

GALE provides more accurate global feature importance than LIME.

Aggregation choice significantly impacts the quality of global explanations.

GALE effectively identifies features that distinguish model behavior.

Abstract

The decision-making process of many state-of-the-art machine learning models is inherently inscrutable to the extent that it is impossible for a human to interpret the model directly: they are black box models. This has led to a call for research on explaining black box models, for which there are two main approaches. Global explanations that aim to explain a model's decision making process in general, and local explanations that aim to explain a single prediction. Since it remains challenging to establish fidelity to black box models in globally interpretable approximations, much attention is put on local explanations. However, whether local explanations are able to reliably represent the black box model and provide useful insights remains an open question. We present Global Aggregations of Local Explanations (GALE) with the objective to provide insights in a model's global decision making process. Overall, our results reveal that the choice of aggregation matters. We find that the global importance introduced by Local Interpretable Model-agnostic Explanations (LIME) does not reliably represent the model's global behavior. Our proposed aggregations are better able to represent how features affect the model's predictions, and to provide global insights by identifying distinguishing features.