Non-linear Learning for Statistical Machine Translation

TL;DR

This paper introduces a neural network-based non-linear approach to model translation hypothesis quality in SMT, surpassing traditional linear models by capturing complex feature interactions.

Contribution

It presents a novel non-linear modeling framework for SMT hypothesis scoring using neural networks, enhancing expressive power over linear models.

Findings

Non-linear models outperform linear models in translation quality.

Neural network-based approach captures complex feature interactions.

Experimental results demonstrate improved translation performance.

Abstract

Modern statistical machine translation (SMT) systems usually use a linear combination of features to model the quality of each translation hypothesis. The linear combination assumes that all the features are in a linear relationship and constrains that each feature interacts with the rest features in an linear manner, which might limit the expressive power of the model and lead to a under-fit model on the current data. In this paper, we propose a non-linear modeling for the quality of translation hypotheses based on neural networks, which allows more complex interaction between features. A learning framework is presented for training the non-linear models. We also discuss possible heuristics in designing the network structure which may improve the non-linear learning performance. Experimental results show that with the basic features of a hierarchical phrase-based machine translation system, our method produce translations that are better than a linear model.