Pluggable Neural Machine Translation Models via Memory-augmented Adapters

TL;DR

This paper introduces a memory-augmented adapter for neural machine translation that allows for customizable, pluggable control over translation style and domain without retraining the entire model, using a novel memory-based approach.

Contribution

It proposes a new memory-augmented adapter architecture and training strategy for flexible, pluggable NMT models that adapt to user-specific requirements efficiently.

Findings

Outperforms several baseline methods in style-specific translation tasks.

Effective in domain adaptation with minimal additional training.

Reduces spurious dependencies through memory dropout.

Abstract

Although neural machine translation (NMT) models perform well in the general domain, it remains rather challenging to control their generation behavior to satisfy the requirement of different users. Given the expensive training cost and the data scarcity challenge of learning a new model from scratch for each user requirement, we propose a memory-augmented adapter to steer pretrained NMT models in a pluggable manner. Specifically, we construct a multi-granular memory based on the user-provided text samples and propose a new adapter architecture to combine the model representations and the retrieved results. We also propose a training strategy using memory dropout to reduce spurious dependencies between the NMT model and the memory. We validate our approach on both style- and domain-specific experiments and the results indicate that our method can outperform several representative pluggable baselines.