Synergy with Translation Artifacts for Training and Inference in Multilingual Tasks

TL;DR

This paper demonstrates that leveraging translation artifacts from both directions in multilingual tasks can enhance performance, introducing novel training methods and a cross-lingual fine-tuning algorithm to capitalize on these artifacts.

Contribution

It is the first to analyze the combined effect of translation artifacts in both training and inference for multilingual tasks, proposing new methods and a fine-tuning algorithm.

Findings

Translation artifacts significantly improve multilingual classification performance.

The proposed MUSC algorithm outperforms existing methods.

Using both translation directions yields synergistic benefits.

Abstract

Translation has played a crucial role in improving the performance on multilingual tasks: (1) to generate the target language data from the source language data for training and (2) to generate the source language data from the target language data for inference. However, prior works have not considered the use of both translations simultaneously. This paper shows that combining them can synergize the results on various multilingual sentence classification tasks. We empirically find that translation artifacts stylized by translators are the main factor of the performance gain. Based on this analysis, we adopt two training methods, SupCon and MixUp, considering translation artifacts. Furthermore, we propose a cross-lingual fine-tuning algorithm called MUSC, which uses SupCon and MixUp jointly and improves the performance. Our code is available at https://github.com/jongwooko/MUSC.