Sparse Regression for Machine Translation

TL;DR

This paper explores using sparse regression techniques, especially Lasso, for learning feature mappings in machine translation, demonstrating improved translation quality and effective training instance selection.

Contribution

It introduces the dice instance selection method and compares L1 and L2 regularized regression for translation mapping tasks, showing L1's superior performance.

Findings

L1 regularized regression outperforms L2 in translation quality.

Proper training instance selection improves feature coverage.

Replacing phrase tables with learned mappings yields promising results.

Abstract

We use transductive regression techniques to learn mappings between source and target features of given parallel corpora and use these mappings to generate machine translation outputs. We show the effectiveness of $L_1$ regularized regression (\textit{lasso}) to learn the mappings between sparsely observed feature sets versus $L_2$ regularized regression. Proper selection of training instances plays an important role to learn correct feature mappings within limited computational resources and at expected accuracy levels. We introduce \textit{dice} instance selection method for proper selection of training instances, which plays an important role to learn correct feature mappings for improving the source and target coverage of the training set. We show that $L_1$ regularized regression performs better than $L_2$ regularized regression both in regression measurements and in the translation experiments using graph decoding. We present encouraging results when translating from German to English and Spanish to English. We also demonstrate results when the phrase table of a phrase-based decoder is replaced with the mappings we find with the regression model.