Explaining Text Similarity in Transformer Models

TL;DR

This paper explores how layer-wise relevance propagation (LRP) and BiLRP can explain the inner workings of Transformer-based similarity models in NLP, providing insights into feature interactions and semantic understanding.

Contribution

It introduces the use of BiLRP for second-order explanations in Transformer similarity models, enabling detailed analysis of feature interactions in NLP tasks.

Findings

BiLRP effectively reveals feature interactions driving similarity.

Explainability methods improve understanding of multilingual semantics.

Insights assist in biomedical text retrieval analysis.

Abstract

As Transformers have become state-of-the-art models for natural language processing (NLP) tasks, the need to understand and explain their predictions is increasingly apparent. Especially in unsupervised applications, such as information retrieval tasks, similarity models built on top of foundation model representations have been widely applied. However, their inner prediction mechanisms have mostly remained opaque. Recent advances in explainable AI have made it possible to mitigate these limitations by leveraging improved explanations for Transformers through layer-wise relevance propagation (LRP). Using BiLRP, an extension developed for computing second-order explanations in bilinear similarity models, we investigate which feature interactions drive similarity in NLP models. We validate the resulting explanations and demonstrate their utility in three corpus-level use cases, analyzing grammatical interactions, multilingual semantics, and biomedical text retrieval. Our findings contribute to a deeper understanding of different semantic similarity tasks and models, highlighting how novel explainable AI methods enable in-depth analyses and corpus-level insights.