Better Explain Transformers by Illuminating Important Information
Linxin Song, Yan Cui, Ao Luo, Freddy Lecue, Irene Li

TL;DR
This paper improves transformer explanations by focusing on important information through a refined relevance propagation method, leading to more accurate and interpretable attributions in NLP tasks.
Contribution
It introduces a masking approach on top of layer-wise relevance propagation, emphasizing important attention heads to enhance explanation quality.
Findings
Outperforms eight baselines on explanation metrics
Achieves 3% to 33% improvement in explanation accuracy
Effectively masks irrelevant information to clarify model decisions
Abstract
Transformer-based models excel in various natural language processing (NLP) tasks, attracting countless efforts to explain their inner workings. Prior methods explain Transformers by focusing on the raw gradient and attention as token attribution scores, where non-relevant information is often considered during explanation computation, resulting in confusing results. In this work, we propose highlighting the important information and eliminating irrelevant information by a refined information flow on top of the layer-wise relevance propagation (LRP) method. Specifically, we consider identifying syntactic and positional heads as important attention heads and focus on the relevance obtained from these important heads. Experimental results demonstrate that irrelevant information does distort output attribution scores and then should be masked during explanation computation. Compared to…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Explainable Artificial Intelligence (XAI) · Machine Learning in Healthcare
MethodsFocus
