Language Modelling Approaches to Adaptive Machine Translation

TL;DR

This paper explores how large language models can enhance adaptive machine translation by enabling real-time, context-aware translation improvements and domain adaptation without extensive in-domain data or fine-tuning.

Contribution

It investigates the use of pre-trained LLMs for improving adaptive MT in low-data scenarios and during human-in-the-loop translation processes.

Findings

LLMs can improve translation consistency with minimal in-domain data

Real-time adaptation using LLMs enhances translation quality during human interaction

Pre-trained models facilitate domain adaptation without additional fine-tuning

Abstract

Consistency is a key requirement of high-quality translation. It is especially important to adhere to pre-approved terminology and adapt to corrected translations in domain-specific projects. Machine translation (MT) has achieved significant progress in the area of domain adaptation. However, in-domain data scarcity is common in translation settings, due to the lack of specialised datasets and terminology, or inconsistency and inaccuracy of available in-domain translations. In such scenarios where there is insufficient in-domain data to fine-tune MT models, producing translations that are consistent with the relevant context is challenging. While real-time adaptation can make use of smaller amounts of in-domain data to improve the translation on the fly, it remains challenging due to supported context limitations and efficiency constraints. Large language models (LLMs) have recently shown interesting capabilities of in-context learning, where they learn to replicate certain input-output text generation patterns, without further fine-tuning. Such capabilities have opened new horizons for domain-specific data augmentation and real-time adaptive MT. This work attempts to address two main relevant questions: 1) in scenarios involving human interaction and continuous feedback, can we employ language models to improve the quality of adaptive MT at inference time? and 2) in the absence of sufficient in-domain data, can we use pre-trained large-scale language models to improve the process of MT domain adaptation?