# Saliency Maps Generation for Automatic Text Summarization

## TL;DR

This paper investigates the effectiveness of saliency map techniques, specifically Layer-Wise Relevance Propagation, in explaining complex text summarization models, highlighting the need for quantitative validation of these explanations.

## Contribution

It applies LRP to a sequence-to-sequence model for text summarization and proposes a protocol for validating the accuracy of saliency maps as explanations.

## Key findings

- Saliency maps sometimes reflect true model input usage
- Saliency maps can be misleading without validation
- A protocol for testing explanation validity is proposed

## Abstract

Saliency map generation techniques are at the forefront of explainable AI literature for a broad range of machine learning applications. Our goal is to question the limits of these approaches on more complex tasks. In this paper we apply Layer-Wise Relevance Propagation (LRP) to a sequence-to-sequence attention model trained on a text summarization dataset. We obtain unexpected saliency maps and discuss the rightfulness of these "explanations". We argue that we need a quantitative way of testing the counterfactual case to judge the truthfulness of the saliency maps. We suggest a protocol to check the validity of the importance attributed to the input and show that the saliency maps obtained sometimes capture the real use of the input features by the network, and sometimes do not. We use this example to discuss how careful we need to be when accepting them as explanation.