Learning aligned embeddings for semi-supervised word translation using Maximum Mean Discrepancy

TL;DR

This paper introduces WAM, an end-to-end method for aligning word embeddings across languages during translation training using MMD, outperforming existing supervised and unsupervised approaches.

Contribution

The paper presents a novel unsupervised approach for word embedding alignment that does not require known word pairs, using a localized MMD constraint during translation training.

Findings

WAM outperforms existing unsupervised methods.

WAM surpasses supervised methods trained on known translations.

Effective alignment during sentence translation training.

Abstract

Word translation is an integral part of language translation. In machine translation, each language is considered a domain with its own word embedding. The alignment between word embeddings allows linking semantically equivalent words in multilingual contexts. Moreover, it offers a way to infer cross-lingual meaning for words without a direct translation. Current methods for word embedding alignment are either supervised, i.e. they require known word pairs, or learn a cross-domain transformation on fixed embeddings in an unsupervised way. Here we propose an end-to-end approach for word embedding alignment that does not require known word pairs. Our method, termed Word Alignment through MMD (WAM), learns embeddings that are aligned during sentence translation training using a localized Maximum Mean Discrepancy (MMD) constraint between the embeddings. We show that our method not only out-performs unsupervised methods, but also supervised methods that train on known word translations.