Vocabulary Selection Strategies for Neural Machine Translation

TL;DR

This paper explores vocabulary selection methods for neural machine translation, significantly reducing decoding and training times with minimal impact on translation accuracy, thereby enhancing efficiency.

Contribution

It introduces context and embedding-based vocabulary selection strategies and analyzes their speed-accuracy trade-offs in neural translation models.

Findings

Decoding time reduced by up to 90% on CPUs.

Training time decreased by 25%.

Minimal accuracy loss observed.

Abstract

Classical translation models constrain the space of possible outputs by selecting a subset of translation rules based on the input sentence. Recent work on improving the efficiency of neural translation models adopted a similar strategy by restricting the output vocabulary to a subset of likely candidates given the source. In this paper we experiment with context and embedding-based selection methods and extend previous work by examining speed and accuracy trade-offs in more detail. We show that decoding time on CPUs can be reduced by up to 90% and training time by 25% on the WMT15 English-German and WMT16 English-Romanian tasks at the same or only negligible change in accuracy. This brings the time to decode with a state of the art neural translation system to just over 140 msec per sentence on a single CPU core for English-German.