Improving Fluency of Non-Autoregressive Machine Translation

TL;DR

This paper enhances the fluency of non-autoregressive machine translation models using CTC and additional features, maintaining high decoding speed and achieving competitive BLEU scores across multiple language pairs.

Contribution

It introduces a method to improve nAR translation fluency with CTC and feature-enhanced scoring, preserving decoding speed advantages.

Findings

Improved fluency of nAR models with CTC and feature scoring.

Decoding speed remains higher than AR models.

Achieved competitive BLEU scores on multiple language pairs.

Abstract

Non-autoregressive (nAR) models for machine translation (MT) manifest superior decoding speed when compared to autoregressive (AR) models, at the expense of impaired fluency of their outputs. We improve the fluency of a nAR model with connectionist temporal classification (CTC) by employing additional features in the scoring model used during beam search decoding. Since the beam search decoding in our model only requires to run the network in a single forward pass, the decoding speed is still notably higher than in standard AR models. We train models for three language pairs: German, Czech, and Romanian from and into English. The results show that our proposed models can be more efficient in terms of decoding speed and still achieve a competitive BLEU score relative to AR models.