Deep Reinforced Self-Attention Masks for Abstractive Summarization (DR.SAS)

TL;DR

This paper introduces DR.SAS, a reinforcement learning-based method that dynamically adjusts self-attention in Transformers to enhance abstractive summarization quality, achieving better ROUGE scores and more factual, coherent summaries.

Contribution

It proposes a novel Actor-Critic reinforcement learning approach to learn dynamic self-attention masks, improving summarization performance over baseline models.

Findings

Achieved better ROUGE scores than baseline models.

Produced more factual and coherent summaries.

Demonstrated the effectiveness of dynamic attention learning.

Abstract

We present a novel architectural scheme to tackle the abstractive summarization problem based on the CNN/DMdataset which fuses Reinforcement Learning (RL) withUniLM, which is a pre-trained Deep Learning Model, to solve various natural language tasks. We have tested the limits of learning fine-grained attention in Transformers to improve the summarization quality. UniLM applies attention to the entire token space in a global fashion. We propose DR.SAS which applies the Actor-Critic (AC) algorithm to learn a dynamic self-attention distribution over the tokens to reduce redundancy and generate factual and coherent summaries to improve the quality of summarization. After performing hyperparameter tuning, we achievedbetter ROUGE results compared to the baseline. Our model tends to be more extractive/factual yet coherent in detail because of optimization over ROUGE rewards. We present detailed error analysis with examples of the strengths and limitations of our model. Our codebase will be publicly available on our GitHub.