Multilingual Neural Machine Translation with Deep Encoder and Multiple Shallow Decoders

TL;DR

This paper explores a multilingual translation model with a deep encoder and multiple shallow decoders, achieving faster inference with minimal quality loss for many-to-one translation, but facing challenges in one-to-many scenarios.

Contribution

It introduces the DEMSD model that uses a deep encoder with multiple shallow decoders tailored for different language subsets, improving speed without quality loss in many-to-one translation.

Findings

Achieves 1.8x speedup with no quality drop in many-to-one translation.

Shallow decoders cause quality drop in one-to-many translation, but DEMSD mitigates this.

Deep encoder with multiple shallow decoders is effective for efficiency in multilingual translation.

Abstract

Recent work in multilingual translation advances translation quality surpassing bilingual baselines using deep transformer models with increased capacity. However, the extra latency and memory costs introduced by this approach may make it unacceptable for efficiency-constrained applications. It has recently been shown for bilingual translation that using a deep encoder and shallow decoder (DESD) can reduce inference latency while maintaining translation quality, so we study similar speed-accuracy trade-offs for multilingual translation. We find that for many-to-one translation we can indeed increase decoder speed without sacrificing quality using this approach, but for one-to-many translation, shallow decoders cause a clear quality drop. To ameliorate this drop, we propose a deep encoder with multiple shallow decoders (DEMSD) where each shallow decoder is responsible for a disjoint subset of target languages. Specifically, the DEMSD model with 2-layer decoders is able to obtain a 1.8x speedup on average compared to a standard transformer model with no drop in translation quality.