Thank you BART! Rewarding Pre-Trained Models Improves Formality Style Transfer

TL;DR

This paper demonstrates that fine-tuning pre-trained models like GPT-2 and BART with reward-based training significantly enhances formality style transfer, especially with limited parallel data, setting new performance benchmarks.

Contribution

It introduces a reward-based fine-tuning approach for pre-trained models that improves content preservation and style transfer quality in formality tasks.

Findings

Reward-based fine-tuning outperforms previous methods

Content preservation is significantly improved

State-of-the-art results achieved with limited data

Abstract

Scarcity of parallel data causes formality style transfer models to have scarce success in preserving content. We show that fine-tuning pre-trained language (GPT-2) and sequence-to-sequence (BART) models boosts content preservation, and that this is possible even with limited amounts of parallel data. Augmenting these models with rewards that target style and content -- the two core aspects of the task -- we achieve a new state-of-the-art.