Faithful Attention Explainer: Verbalizing Decisions Based on Discriminative Features

TL;DR

The paper introduces Faithful Attention Explainer (FAE), a framework that generates faithful textual explanations of model decisions by leveraging attention mechanisms and associating features with words, applicable to visual and gaze-based data.

Contribution

FAE is a novel framework that produces faithful textual explanations by enforcing attention alignment and associating features with words, enhancing interpretability of model decisions.

Findings

Achieves promising caption quality metrics

Demonstrates faithful decision-relevance on datasets

Interprets gaze-based human attention effectively

Abstract

In recent years, model explanation methods have been designed to interpret model decisions faithfully and intuitively so that users can easily understand them. In this paper, we propose a framework, Faithful Attention Explainer (FAE), capable of generating faithful textual explanations regarding the attended-to features. Towards this goal, we deploy an attention module that takes the visual feature maps from the classifier for sentence generation. Furthermore, our method successfully learns the association between features and words, which allows a novel attention enforcement module for attention explanation. Our model achieves promising performance in caption quality metrics and a faithful decision-relevance metric on two datasets (CUB and ACT-X). In addition, we show that FAE can interpret gaze-based human attention, as human gaze indicates the discriminative features that humans use for decision-making, demonstrating the potential of deploying human gaze for advanced human-AI interaction.