Exploring Cross-lingual Textual Style Transfer with Large Multilingual Language Models

TL;DR

This paper investigates the ability of large multilingual models to perform detoxification across multiple languages without fine-tuning, revealing their strengths in multilingual style transfer but limitations in cross-lingual detoxification.

Contribution

It explores cross-lingual detoxification with large models and demonstrates the necessity of language-specific fine-tuning for effective detoxification.

Findings

Multilingual models can perform style transfer across languages.

Models struggle with cross-lingual detoxification without fine-tuning.

Fine-tuning on specific languages remains necessary for detoxification.

Abstract

Detoxification is a task of generating text in polite style while preserving meaning and fluency of the original toxic text. Existing detoxification methods are designed to work in one exact language. This work investigates multilingual and cross-lingual detoxification and the behavior of large multilingual models like in this setting. Unlike previous works we aim to make large language models able to perform detoxification without direct fine-tuning in given language. Experiments show that multilingual models are capable of performing multilingual style transfer. However, models are not able to perform cross-lingual detoxification and direct fine-tuning on exact language is inevitable.