Statistical Machine Translation Features with Multitask Tensor Networks

TL;DR

This paper introduces a novel neural network-based approach with tensor layers and multitask learning to enhance statistical machine translation, achieving significant BLEU score improvements for Arabic-English and Chinese-English translations.

Contribution

It presents new neural network features, tensor layer architecture, and multitask training methods specifically designed for SMT, advancing the state-of-the-art.

Findings

+2.7 BLEU points for Arabic-English translation

+1.8 BLEU points for Chinese-English translation

Significant improvements over existing neural network-based SMT systems

Abstract

We present a three-pronged approach to improving Statistical Machine Translation (SMT), building on recent success in the application of neural networks to SMT. First, we propose new features based on neural networks to model various non-local translation phenomena. Second, we augment the architecture of the neural network with tensor layers that capture important higher-order interaction among the network units. Third, we apply multitask learning to estimate the neural network parameters jointly. Each of our proposed methods results in significant improvements that are complementary. The overall improvement is +2.7 and +1.8 BLEU points for Arabic-English and Chinese-English translation over a state-of-the-art system that already includes neural network features.