Multilingual Pre-training with Language and Task Adaptation for Multilingual Text Style Transfer

TL;DR

This paper leverages mBART for multilingual text style transfer, combining machine translation and aligned data to achieve state-of-the-art results, and proposes a modular adaptation approach effective across languages and styles.

Contribution

It introduces a modular, adaptable training strategy for multilingual style transfer that does not rely on monolingual parallel data, enhancing applicability across languages and tasks.

Findings

Achieves state-of-the-art results in three languages.

Effective without monolingual parallel data.

Applicable to various style transfer tasks.

Abstract

We exploit the pre-trained seq2seq model mBART for multilingual text style transfer. Using machine translated data as well as gold aligned English sentences yields state-of-the-art results in the three target languages we consider. Besides, in view of the general scarcity of parallel data, we propose a modular approach for multilingual formality transfer, which consists of two training strategies that target adaptation to both language and task. Our approach achieves competitive performance without monolingual task-specific parallel data and can be applied to other style transfer tasks as well as to other languages.