Faster Machine Translation Ensembling with Reinforcement Learning and Competitive Correction

TL;DR

This paper presents SmartGen, a reinforcement learning-based ensembling method for neural machine translation that reduces computational costs and improves translation quality by selecting optimal candidate groups and incorporating a competitive correction mechanism.

Contribution

Introduces SmartGen, a novel RL-based ensembling strategy with candidate selection and correction, trained jointly for better NMT performance and efficiency.

Findings

Reduces inference time by selecting fewer candidates.

Improves translation quality over baseline ensembling methods.

Validated on English-Hindi translation tasks with positive results.

Abstract

Ensembling neural machine translation (NMT) models to produce higher-quality translations than the $L$ individual models has been extensively studied. Recent methods typically employ a candidate selection block (CSB) and an encoder-decoder fusion block (FB), requiring inference across \textit{all} candidate models, leading to significant computational overhead, generally $\Omega(L)$. This paper introduces \textbf{SmartGen}, a reinforcement learning (RL)-based strategy that improves the CSB by selecting a small, fixed number of candidates and identifying optimal groups to pass to the fusion block for each input sentence. Furthermore, previously, the CSB and FB were trained independently, leading to suboptimal NMT performance. Our DQN-based \textbf{SmartGen} addresses this by using feedback from the FB block as a reward during training. We also resolve a key issue in earlier methods, where candidates were passed to the FB without modification, by introducing a Competitive Correction Block (CCB). Finally, we validate our approach with extensive experiments on English-Hindi translation tasks in both directions.