Deep Communicating Agents for Abstractive Summarization

TL;DR

This paper introduces deep communicating agents within an encoder-decoder framework to improve abstractive summarization of long documents by dividing encoding tasks among multiple collaborating agents, resulting in higher quality summaries.

Contribution

It proposes a novel deep communicating agents architecture that enhances long document encoding through collaboration, trained end-to-end with reinforcement learning.

Findings

Multiple communicating encoders outperform non-communicating ones.

The approach yields higher quality summaries than strong baselines.

End-to-end training improves coherence and focus in summaries.

Abstract

We present deep communicating agents in an encoder-decoder architecture to address the challenges of representing a long document for abstractive summarization. With deep communicating agents, the task of encoding a long text is divided across multiple collaborating agents, each in charge of a subsection of the input text. These encoders are connected to a single decoder, trained end-to-end using reinforcement learning to generate a focused and coherent summary. Empirical results demonstrate that multiple communicating encoders lead to a higher quality summary compared to several strong baselines, including those based on a single encoder or multiple non-communicating encoders.