Towards Robust k-Nearest-Neighbor Machine Translation

TL;DR

This paper introduces a confidence-enhanced kNN-MT model that improves translation quality and robustness by leveraging NMT confidence and perturbation-based training, addressing noise issues in retrieval-based translation methods.

Contribution

It proposes a novel confidence-aware approach with robust training techniques to enhance kNN-MT performance and robustness against noisy retrieved pairs.

Findings

Significant performance improvements over existing kNN-MT models

Enhanced robustness to noisy retrieval pairs

Effective use of NMT confidence in model components

Abstract

k-Nearest-Neighbor Machine Translation (kNN-MT) becomes an important research direction of NMT in recent years. Its main idea is to retrieve useful key-value pairs from an additional datastore to modify translations without updating the NMT model. However, the underlying retrieved noisy pairs will dramatically deteriorate the model performance. In this paper, we conduct a preliminary study and find that this problem results from not fully exploiting the prediction of the NMT model. To alleviate the impact of noise, we propose a confidence-enhanced kNN-MT model with robust training. Concretely, we introduce the NMT confidence to refine the modeling of two important components of kNN-MT: kNN distribution and the interpolation weight. Meanwhile we inject two types of perturbations into the retrieved pairs for robust training. Experimental results on four benchmark datasets demonstrate that our model not only achieves significant improvements over current kNN-MT models, but also exhibits better robustness. Our code is available at https://github.com/DeepLearnXMU/Robust-knn-mt.