Towards Document-Level Paraphrase Generation with Sentence Rewriting and Reordering

TL;DR

This paper introduces CoRPG, a novel method for document-level paraphrase generation that uses graph GRU to encode coherence and improve diversity and semantic preservation, addressing a previously underexplored task.

Contribution

It pioneers the exploration of document-level paraphrase generation and proposes a coherence-aware model leveraging graph GRU for sentence rewriting and reordering.

Findings

CoRPG outperforms baseline models on BERTScore and diversity metrics.

Human evaluation confirms higher diversity and semantic preservation.

Created a pseudo dataset for training document-level paraphrasing models.

Abstract

Paraphrase generation is an important task in natural language processing. Previous works focus on sentence-level paraphrase generation, while ignoring document-level paraphrase generation, which is a more challenging and valuable task. In this paper, we explore the task of document-level paraphrase generation for the first time and focus on the inter-sentence diversity by considering sentence rewriting and reordering. We propose CoRPG (Coherence Relationship guided Paraphrase Generation), which leverages graph GRU to encode the coherence relationship graph and get the coherence-aware representation for each sentence, which can be used for re-arranging the multiple (possibly modified) input sentences. We create a pseudo document-level paraphrase dataset for training CoRPG. Automatic evaluation results show CoRPG outperforms several strong baseline models on the BERTScore and diversity scores. Human evaluation also shows our model can generate document paraphrase with more diversity and semantic preservation.