Learning Bilingual Word Representations by Marginalizing Alignments

TL;DR

This paper introduces a probabilistic model that learns bilingual word representations by marginalizing over alignments, capturing broader semantic context and improving cross-lingual classification performance.

Contribution

It proposes a novel probabilistic approach that marginalizes alignments to learn richer bilingual word embeddings, outperforming previous methods.

Findings

Outperforms previous state-of-the-art in cross-lingual classification

Captures larger semantic context than hard alignment models

Demonstrates effectiveness of marginalized alignment approach

Abstract

We present a probabilistic model that simultaneously learns alignments and distributed representations for bilingual data. By marginalizing over word alignments the model captures a larger semantic context than prior work relying on hard alignments. The advantage of this approach is demonstrated in a cross-lingual classification task, where we outperform the prior published state of the art.