Distributionally Robust Multilingual Machine Translation

TL;DR

This paper introduces a distributionally robust optimization approach for multilingual neural machine translation, improving performance across language pairs despite data imbalance issues.

Contribution

It proposes a novel learning objective for MNMT based on distributionally robust optimization, with an efficient optimization scheme for large datasets.

Findings

Consistently outperforms baseline methods in translation quality.

Effective in both many-to-one and one-to-many translation scenarios.

Achieves improved per-language performance across multiple datasets.

Abstract

Multilingual neural machine translation (MNMT) learns to translate multiple language pairs with a single model, potentially improving both the accuracy and the memory-efficiency of deployed models. However, the heavy data imbalance between languages hinders the model from performing uniformly across language pairs. In this paper, we propose a new learning objective for MNMT based on distributionally robust optimization, which minimizes the worst-case expected loss over the set of language pairs. We further show how to practically optimize this objective for large translation corpora using an iterated best response scheme, which is both effective and incurs negligible additional computational cost compared to standard empirical risk minimization. We perform extensive experiments on three sets of languages from two datasets and show that our method consistently outperforms strong baseline methods in terms of average and per-language performance under both many-to-one and one-to-many translation settings.