# Generalizing Back-Translation in Neural Machine Translation

## TL;DR

This paper provides a mathematical reformulation of back-translation in neural machine translation, broadening its scope and addressing fundamental issues in sampling-based methods to improve translation quality.

## Contribution

It introduces a formal framework for back-translation, analyzes its assumptions, and proposes practical modifications to enhance sampling-based synthetic data generation.

## Key findings

- Disabling label smoothing improves sampling quality.
- Sampling from a restricted search space reduces errors.
- The reformulation clarifies the theoretical basis of back-translation.

## Abstract

Back-translation - data augmentation by translating target monolingual data - is a crucial component in modern neural machine translation (NMT). In this work, we reformulate back-translation in the scope of cross-entropy optimization of an NMT model, clarifying its underlying mathematical assumptions and approximations beyond its heuristic usage. Our formulation covers broader synthetic data generation schemes, including sampling from a target-to-source NMT model. With this formulation, we point out fundamental problems of the sampling-based approaches and propose to remedy them by (i) disabling label smoothing for the target-to-source model and (ii) sampling from a restricted search space. Our statements are investigated on the WMT 2018 German - English news translation task.