Wasserstein distances for evaluating cross-lingual embeddings

TL;DR

This paper introduces a novel evaluation method for cross-lingual word embeddings using Wasserstein distances within an optimal transport framework, demonstrating improved performance on retrieval and classification tasks.

Contribution

It adapts the Wasserstein distance to evaluate cross-lingual embeddings directly through downstream NLP tasks, offering a new perspective on embedding quality assessment.

Findings

Wasserstein distances outperform traditional evaluation methods.

The approach achieves competitive results with state-of-the-art models.

It effectively solves cross-lingual document retrieval and classification.

Abstract

Word embeddings are high dimensional vector representations of words that capture their semantic similarity in the vector space. There exist several algorithms for learning such embeddings both for a single language as well as for several languages jointly. In this work we propose to evaluate collections of embeddings by adapting downstream natural language tasks to the optimal transport framework. We show how the family of Wasserstein distances can be used to solve cross-lingual document retrieval and the cross-lingual document classification problems. We argue on the advantages of this approach compared to more traditional evaluation methods of embeddings like bilingual lexical induction. Our experimental results suggest that using Wasserstein distances on these problems out-performs several strong baselines and performs on par with state-of-the-art models.