Disentangling Specificity for Abstractive Multi-document Summarization

TL;DR

This paper introduces a method to disentangle document-specific content from shared information in multi-document summarization, enhancing summary comprehensiveness by explicitly modeling unique document details.

Contribution

It proposes a novel disentanglement approach with an orthogonal constraint to better capture document-specific information in MDS.

Findings

Disentangling specific content improves summary quality.

Shared information contributes less to MDS performance.

Combining specific and shared representations yields more comprehensive summaries.

Abstract

Multi-document summarization (MDS) generates a summary from a document set. Each document in a set describes topic-relevant concepts, while per document also has its unique contents. However, the document specificity receives little attention from existing MDS approaches. Neglecting specific information for each document limits the comprehensiveness of the generated summaries. To solve this problem, in this paper, we propose to disentangle the specific content from documents in one document set. The document-specific representations, which are encouraged to be distant from each other via a proposed orthogonal constraint, are learned by the specific representation learner. We provide extensive analysis and have interesting findings that specific information and document set representations contribute distinctive strengths and their combination yields a more comprehensive solution for the MDS. Also, we find that the common (i.e. shared) information could not contribute much to the overall performance under the MDS settings. Implemetation codes are available at https://github.com/congboma/DisentangleSum.