Relational Local Explanations

TL;DR

This paper introduces a novel, model-agnostic, permutation-based feature attribution method that considers inter-variable relationships, providing deeper insights into model predictions for structured data like images and text.

Contribution

It proposes a new relational analysis-based attribution approach that captures latent feature relationships, improving interpretability over traditional independent attribution methods.

Findings

Outperforms state-of-the-art attribution techniques on image and text data.

Provides broader insights into model decision-making processes.

Validated through extensive experimental evaluations.

Abstract

The majority of existing post-hoc explanation approaches for machine learning models produce independent, per-variable feature attribution scores, ignoring a critical inherent characteristics of homogeneously structured data, such as visual or text data: there exist latent inter-variable relationships between features. In response, we develop a novel model-agnostic and permutation-based feature attribution approach based on the relational analysis between input variables. As a result, we are able to gain a broader insight into the predictions and decisions of machine learning models. Experimental evaluations of our framework in comparison with state-of-the-art attribution techniques on various setups involving both image and text data modalities demonstrate the effectiveness and validity of our method.