Trans-gram, Fast Cross-lingual Word-embeddings

TL;DR

Trans-gram is a computationally-efficient method for learning and aligning multilingual word embeddings using monolingual and limited sentence-aligned data, enabling cross-lingual tasks without extensive aligned datasets.

Contribution

It introduces Trans-gram, a novel approach that efficiently aligns word embeddings across multiple languages with minimal aligned data, outperforming existing methods.

Findings

Aligned embeddings for 21 languages using English as pivot

Achieved state-of-the-art results on cross-lingual classification

Discovered linguistic features aligned across languages without direct data

Abstract

We introduce Trans-gram, a simple and computationally-efficient method to simultaneously learn and align wordembeddings for a variety of languages, using only monolingual data and a smaller set of sentence-aligned data. We use our new method to compute aligned wordembeddings for twenty-one languages using English as a pivot language. We show that some linguistic features are aligned across languages for which we do not have aligned data, even though those properties do not exist in the pivot language. We also achieve state of the art results on standard cross-lingual text classification and word translation tasks.