Translating Pro-Drop Languages with Reconstruction Models

TL;DR

This paper introduces a reconstruction-based method to improve neural machine translation of pro-drop languages by explicitly handling dropped pronouns, leading to better translation quality in Chinese-English and Japanese-English tasks.

Contribution

It proposes a novel approach that automatically annotates dropped pronouns and guides NMT models to better incorporate this information through reconstruction objectives.

Findings

Significant improvement over baseline NMT models.

Effective handling of dropped pronouns in Chinese-English and Japanese-English translation.

Consistent performance gains across multiple datasets.

Abstract

Pronouns are frequently omitted in pro-drop languages, such as Chinese, generally leading to significant challenges with respect to the production of complete translations. To date, very little attention has been paid to the dropped pronoun (DP) problem within neural machine translation (NMT). In this work, we propose a novel reconstruction-based approach to alleviating DP translation problems for NMT models. Firstly, DPs within all source sentences are automatically annotated with parallel information extracted from the bilingual training corpus. Next, the annotated source sentence is reconstructed from hidden representations in the NMT model. With auxiliary training objectives, in terms of reconstruction scores, the parameters associated with the NMT model are guided to produce enhanced hidden representations that are encouraged as much as possible to embed annotated DP information. Experimental results on both Chinese-English and Japanese-English dialogue translation tasks show that the proposed approach significantly and consistently improves translation performance over a strong NMT baseline, which is directly built on the training data annotated with DPs.