Is Neural Machine Translation Ready for Deployment? A Case Study on 30 Translation Directions

TL;DR

This study compares phrase-based SMT, hierarchical phrase-based MT, and neural machine translation across 30 language pairs, evaluating quality and speed, and introduces AmuNMT, an efficient decoder, showing NMT's readiness for deployment.

Contribution

It provides the largest comparison of translation quality across multiple MT paradigms and introduces AmuNMT, a fast neural decoder suitable for production use.

Findings

Neural machine translation performs comparably to phrase-based SMT in many directions.

AmuNMT significantly improves translation speed, enabling real-time deployment.

NMT is feasible for in-production systems based on speed and quality metrics.

Abstract

In this paper we provide the largest published comparison of translation quality for phrase-based SMT and neural machine translation across 30 translation directions. For ten directions we also include hierarchical phrase-based MT. Experiments are performed for the recently published United Nations Parallel Corpus v1.0 and its large six-way sentence-aligned subcorpus. In the second part of the paper we investigate aspects of translation speed, introducing AmuNMT, our efficient neural machine translation decoder. We demonstrate that current neural machine translation could already be used for in-production systems when comparing words-per-second ratios.