Towards Continual Learning for Multilingual Machine Translation via Vocabulary Substitution

TL;DR

This paper introduces a simple vocabulary adaptation method to enhance multilingual machine translation models, enabling efficient continual learning across languages, including distant and unseen scripts, with minimal performance loss.

Contribution

It presents a novel vocabulary substitution scheme that allows scalable, effective continual learning for multilingual translation, even with limited data for new languages.

Findings

Minor degradation on original language pairs

Effective for distant languages with unseen scripts

Competitive performance with only monolingual data

Abstract

We propose a straightforward vocabulary adaptation scheme to extend the language capacity of multilingual machine translation models, paving the way towards efficient continual learning for multilingual machine translation. Our approach is suitable for large-scale datasets, applies to distant languages with unseen scripts, incurs only minor degradation on the translation performance for the original language pairs and provides competitive performance even in the case where we only possess monolingual data for the new languages.