Simply Trainable Nearest Neighbour Machine Translation with GPU Inference

TL;DR

This paper introduces a GPU-efficient, trainable nearest neighbor machine translation method that adaptively constructs small datastores and learns interpolation coefficients, improving translation quality across domains with minimal speed loss.

Contribution

It proposes a simple, trainable interpolation approach for kNN-based machine translation that is automatic and GPU-compatible, enhancing domain adaptation without retraining the entire model.

Findings

Improves or maintains translation quality across domains.

Achieves only 5% speed reduction on GPU inference.

Automatically learns interpolation coefficients for better domain adaptation.

Abstract

Nearest neighbor machine translation is a successful approach for fast domain adaption, which interpolates the pre-trained transformers with domain-specific token-level k-nearest-neighbor (kNN) retrieval without retraining. Despite kNN MT's success, searching large reference corpus and fixed interpolation between the kNN and pre-trained model led to computational complexity and translation quality challenges. Among other papers, Dai et al. proposed methods to obtain a small number of reference samples dynamically for which they introduced a distance-aware interpolation method using an equation that includes free parameters. This paper proposes a simply trainable nearest neighbor machine translation and carry out inference experiments on GPU. Similar to Dai et al., we first adaptively construct a small datastore for each input sentence. Second, we train a single-layer network for the interpolation coefficient between the knnMT and pre-trained result to automatically interpolate in different domains. Experimental results on different domains show that our proposed method either improves or sometimes maintain the translation quality of methods in Dai et al. while being automatic. In addition, our GPU inference results demonstrate that knnMT can be integrated into GPUs with a drop of only 5% in terms of speed.