Reference Network for Neural Machine Translation

TL;DR

This paper introduces a Reference Network for NMT that incorporates referencing into translation decoding using Local Coordinates Coding, improving translation quality efficiently.

Contribution

It proposes a novel Reference Network with LCC for context-aware NMT decoding, reducing computational cost while enhancing translation accuracy.

Findings

Improved translation quality on Chinese-English and English-German tasks.

Efficient referencing mechanism with lightweight computation.

Effective integration of monolingual and bilingual context information.

Abstract

Neural Machine Translation (NMT) has achieved notable success in recent years. Such a framework usually generates translations in isolation. In contrast, human translators often refer to reference data, either rephrasing the intricate sentence fragments with common terms in source language, or just accessing to the golden translation directly. In this paper, we propose a Reference Network to incorporate referring process into translation decoding of NMT. To construct a \emph{reference book}, an intuitive way is to store the detailed translation history with extra memory, which is computationally expensive. Instead, we employ Local Coordinates Coding (LCC) to obtain global context vectors containing monolingual and bilingual contextual information for NMT decoding. Experimental results on Chinese-English and English-German tasks demonstrate that our proposed model is effective in improving the translation quality with lightweight computation cost.