DialUp! Modeling the Language Continuum by Adapting Models to Dialects and Dialects to Models

TL;DR

This paper introduces DialUp, a dual approach to improve machine translation for low-resource dialects by adapting models during training and inference, leveraging linguistic regularities and synthetic data.

Contribution

The paper presents DialUp, a novel method combining training-time and inference-time adaptations to enhance MT robustness to dialectal variation.

Findings

Significant performance improvements across multiple dialects and language families.

Synthetic data exposure enhances model robustness to unseen dialects.

Low baseline MT performance varieties benefit most from these methods.

Abstract

Most of the world's languages and dialects are low-resource, and lack support in mainstream machine translation (MT) models. However, many of them have a closely-related high-resource language (HRL) neighbor, and differ in linguistically regular ways from it. This underscores the importance of model robustness to dialectal variation and cross-lingual generalization to the HRL dialect continuum. We present DialUp, consisting of a training-time technique for adapting a pretrained model to dialectal data (M->D), and an inference-time intervention adapting dialectal data to the model expertise (D->M). M->D induces model robustness to potentially unseen and unknown dialects by exposure to synthetic data exemplifying linguistic mechanisms of dialectal variation, whereas D->M treats dialectal divergence for known target dialects. These methods show considerable performance gains for several dialects from four language families, and modest gains for two other language families. We also conduct feature and error analyses, which show that language varieties with low baseline MT performance are more likely to benefit from these approaches.