Analyzing the Use of Influence Functions for Instance-Specific Data Filtering in Neural Machine Translation

TL;DR

This paper explores the application of influence functions to identify and filter training data in neural machine translation, aiming to improve translation quality by removing problematic instances.

Contribution

It extends influence functions for NMT and demonstrates their effectiveness in filtering copied training examples beyond traditional regex methods.

Findings

Influence functions can identify influential training examples in NMT.

Extensions to influence functions improve their applicability and effectiveness.

Filtering training data with influence functions enhances translation quality.

Abstract

Customer feedback can be an important signal for improving commercial machine translation systems. One solution for fixing specific translation errors is to remove the related erroneous training instances followed by re-training of the machine translation system, which we refer to as instance-specific data filtering. Influence functions (IF) have been shown to be effective in finding such relevant training examples for classification tasks such as image classification, toxic speech detection and entailment task. Given a probing instance, IF find influential training examples by measuring the similarity of the probing instance with a set of training examples in gradient space. In this work, we examine the use of influence functions for Neural Machine Translation (NMT). We propose two effective extensions to a state of the art influence function and demonstrate on the sub-problem of copied training examples that IF can be applied more generally than handcrafted regular expressions.