Accelerating NMT Batched Beam Decoding with LMBR Posteriors for Deployment

TL;DR

This paper introduces a batched beam decoding method for neural machine translation that incorporates LMBR posteriors, improving performance and deployment efficiency over existing Transformer-based models.

Contribution

It presents a novel batched decoding algorithm with LMBR posteriors and discusses acceleration strategies for practical deployment, enhancing NMT decoding speed and memory efficiency.

Findings

LMBR posteriors improve translation quality beyond Transformer baselines

Batched decoding accelerates inference without sacrificing accuracy

Deployment strategies reduce memory usage and increase speed

Abstract

We describe a batched beam decoding algorithm for NMT with LMBR n-gram posteriors, showing that LMBR techniques still yield gains on top of the best recently reported results with Transformers. We also discuss acceleration strategies for deployment, and the effect of the beam size and batching on memory and speed.