Evaluating Input Feature Explanations through a Unified Diagnostic Evaluation Framework

TL;DR

This paper introduces a unified framework for comparing different input feature explanation methods in machine learning, revealing that interactive span explanations generally outperform other types across various diagnostic properties.

Contribution

The authors develop a comprehensive framework for directly comparing highlight and interactive explanations, enabling systematic evaluation across multiple diagnostic properties.

Findings

Interactive span explanations outperform other explanation types in most diagnostic properties.

Different explanation methods have distinct strengths depending on the diagnostic property.

The study highlights the need for further research to improve and combine explanation techniques.

Abstract

Explaining the decision-making process of machine learning models is crucial for ensuring their reliability and transparency for end users. One popular explanation form highlights key input features, such as i) tokens (e.g., Shapley Values and Integrated Gradients), ii) interactions between tokens (e.g., Bivariate Shapley and Attention-based methods), or iii) interactions between spans of the input (e.g., Louvain Span Interactions). However, these explanation types have only been studied in isolation, making it difficult to judge their respective applicability. To bridge this gap, we develop a unified framework that facilitates an automated and direct comparison between highlight and interactive explanations comprised of four diagnostic properties. We conduct an extensive analysis across these three types of input feature explanations -- each utilizing three different explanation techniques -- across two datasets and two models, and reveal that each explanation has distinct strengths across the different diagnostic properties. Nevertheless, interactive span explanations outperform other types of input feature explanations across most diagnostic properties. Despite being relatively understudied, our analysis underscores the need for further research to improve methods generating these explanation types. Additionally, integrating them with other explanation types that perform better in certain characteristics could further enhance their overall effectiveness.