Multilingual Word Embeddings using Multigraphs

TL;DR

This paper introduces neural-network-inspired models for creating multilingual word embeddings that leverage monolingual and multilingual text, improving semantic understanding and translation quality in an unsupervised manner.

Contribution

It proposes a novel framework for unsupervised training of multilingual embeddings that enhances syntactic, semantic, and cross-lingual tasks over previous models.

Findings

Embeddings outperform previous models in syntactic and semantic tasks.

Multilingual embeddings improve statistical machine translation for unseen words.

Unsupervised training achieves high accuracy without labeled data.

Abstract

We present a family of neural-network--inspired models for computing continuous word representations, specifically designed to exploit both monolingual and multilingual text. This framework allows us to perform unsupervised training of embeddings that exhibit higher accuracy on syntactic and semantic compositionality, as well as multilingual semantic similarity, compared to previous models trained in an unsupervised fashion. We also show that such multilingual embeddings, optimized for semantic similarity, can improve the performance of statistical machine translation with respect to how it handles words not present in the parallel data.