Contrastive Representation Learning for Exemplar-Guided Paraphrase Generation

TL;DR

This paper introduces a contrastive learning approach for exemplar-guided paraphrase generation, improving style and content representation by using dual contrastive losses within an encoder-decoder framework.

Contribution

It proposes a novel contrastive learning method with two losses to better capture style and content in paraphrase generation, demonstrating effectiveness on benchmark datasets.

Findings

Enhanced style and content representation in paraphrases

Improved performance on QQP-Pos and ParaNMT datasets

Effective integration of contrastive losses into encoder-decoder models

Abstract

Exemplar-Guided Paraphrase Generation (EGPG) aims to generate a target sentence which conforms to the style of the given exemplar while encapsulating the content information of the source sentence. In this paper, we propose a new method with the goal of learning a better representation of the style andthe content. This method is mainly motivated by the recent success of contrastive learning which has demonstrated its power in unsupervised feature extraction tasks. The idea is to design two contrastive losses with respect to the content and the style by considering two problem characteristics during training. One characteristic is that the target sentence shares the same content with the source sentence, and the second characteristic is that the target sentence shares the same style with the exemplar. These two contrastive losses are incorporated into the general encoder-decoder paradigm. Experiments on two datasets, namely QQP-Pos and ParaNMT, demonstrate the effectiveness of our proposed constrastive losses.