XFORMAL: A Benchmark for Multilingual Formality Style Transfer

TL;DR

XFORMAL introduces a multilingual benchmark for formal style transfer across Brazilian Portuguese, French, and Italian, revealing current models' limitations and highlighting the challenge of multilingual style transfer.

Contribution

This paper presents the first multilingual style transfer benchmark, XFORMAL, with datasets in three languages, enabling evaluation of style transfer methods across diverse languages.

Findings

State-of-the-art models perform similarly to simple baselines on XFORMAL.

Multilingual style transfer remains a challenging task.

Current approaches need improvement for effective multilingual style transfer.

Abstract

We take the first step towards multilingual style transfer by creating and releasing XFORMAL, a benchmark of multiple formal reformulations of informal text in Brazilian Portuguese, French, and Italian. Results on XFORMAL suggest that state-of-the-art style transfer approaches perform close to simple baselines, indicating that style transfer is even more challenging when moving multilingual.