An Attention Matrix for Every Decision: Faithfulness-based Arbitration Among Multiple Attention-Based Interpretations of Transformers in Text Classification

TL;DR

This paper introduces a faithfulness-based arbitration method to select the most interpretable attention-based explanation in transformer models for text classification, improving interpretability and efficiency.

Contribution

It proposes a novel arbitration technique for selecting the most faithful attention interpretation, along with two efficiency-enhancing variations and a new faithfulness metric for transformers.

Findings

The method effectively identifies the most faithful attention interpretation.

The proposed variations reduce computational complexity and improve multi-label performance.

The new faithfulness metric correlates well with ground truth rationales.

Abstract

Transformers are widely used in natural language processing, where they consistently achieve state-of-the-art performance. This is mainly due to their attention-based architecture, which allows them to model rich linguistic relations between (sub)words. However, transformers are difficult to interpret. Being able to provide reasoning for its decisions is an important property for a model in domains where human lives are affected. With transformers finding wide use in such fields, the need for interpretability techniques tailored to them arises. We propose a new technique that selects the most faithful attention-based interpretation among the several ones that can be obtained by combining different head, layer and matrix operations. In addition, two variations are introduced towards (i) reducing the computational complexity, thus being faster and friendlier to the environment, and (ii) enhancing the performance in multi-label data. We further propose a new faithfulness metric that is more suitable for transformer models and exhibits high correlation with the area under the precision-recall curve based on ground truth rationales. We validate the utility of our contributions with a series of quantitative and qualitative experiments on seven datasets.