Single Model Ensemble for Subword Regularized Models in Low-Resource Machine Translation

TL;DR

This paper introduces a single-model ensemble inference strategy for subword regularized neural machine translation, improving performance in low-resource settings without additional training costs.

Contribution

It proposes a novel inference method that aggregates multiple segmentations, effectively creating an ensemble from a single model, enhancing robustness in low-resource translation.

Findings

Improved translation accuracy in low-resource scenarios.

Effective ensemble approximation without extra training.

Enhanced robustness through multiple segmentation aggregation.

Abstract

Subword regularizations use multiple subword segmentations during training to improve the robustness of neural machine translation models. In previous subword regularizations, we use multiple segmentations in the training process but use only one segmentation in the inference. In this study, we propose an inference strategy to address this discrepancy. The proposed strategy approximates the marginalized likelihood by using multiple segmentations including the most plausible segmentation and several sampled segmentations. Because the proposed strategy aggregates predictions from several segmentations, we can regard it as a single model ensemble that does not require any additional cost for training. Experimental results show that the proposed strategy improves the performance of models trained with subword regularization in low-resource machine translation tasks.