Iterative Refinement in the Continuous Space for Non-Autoregressive Neural Machine Translation

TL;DR

This paper introduces an efficient continuous-space iterative refinement method for non-autoregressive neural machine translation, improving speed and translation quality over previous approaches by optimizing in a low-dimensional latent space.

Contribution

The authors propose a novel gradient-based inference procedure in continuous latent space for non-autoregressive translation, outperforming existing hybrid-space methods in efficiency and effectiveness.

Findings

Faster decoding compared to EM-like inference methods.

Higher BLEU scores with the same number of refinement steps.

Decodes 6.2 times faster than autoregressive models with minimal quality loss.

Abstract

We propose an efficient inference procedure for non-autoregressive machine translation that iteratively refines translation purely in the continuous space. Given a continuous latent variable model for machine translation (Shu et al., 2020), we train an inference network to approximate the gradient of the marginal log probability of the target sentence, using only the latent variable as input. This allows us to use gradient-based optimization to find the target sentence at inference time that approximately maximizes its marginal probability. As each refinement step only involves computation in the latent space of low dimensionality (we use 8 in our experiments), we avoid computational overhead incurred by existing non-autoregressive inference procedures that often refine in token space. We compare our approach to a recently proposed EM-like inference procedure (Shu et al., 2020) that optimizes in a hybrid space, consisting of both discrete and continuous variables. We evaluate our approach on WMT'14 En-De, WMT'16 Ro-En and IWSLT'16 De-En, and observe two advantages over the EM-like inference: (1) it is computationally efficient, i.e. each refinement step is twice as fast, and (2) it is more effective, resulting in higher marginal probabilities and BLEU scores with the same number of refinement steps. On WMT'14 En-De, for instance, our approach is able to decode 6.2 times faster than the autoregressive model with minimal degradation to translation quality (0.9 BLEU).