Cross-lingual Word Embeddings in Hyperbolic Space

TL;DR

This paper introduces a hyperbolic space-based cross-lingual Word2Vec model that captures hierarchical relationships across languages, showing comparable performance on analogy tasks and improved hierarchical structure representation.

Contribution

It presents the first adaptation of Word2Vec to hyperbolic space for cross-lingual embeddings, capturing hierarchical structures better than Euclidean models.

Findings

Hyperbolic embeddings preserve hierarchical relationships.

The model performs comparably on analogy tasks.

Hyperbolic space captures latent hierarchical structures.

Abstract

Cross-lingual word embeddings can be applied to several natural language processing applications across multiple languages. Unlike prior works that use word embeddings based on the Euclidean space, this short paper presents a simple and effective cross-lingual Word2Vec model that adapts to the Poincar\'e ball model of hyperbolic space to learn unsupervised cross-lingual word representations from a German-English parallel corpus. It has been shown that hyperbolic embeddings can capture and preserve hierarchical relationships. We evaluate the model on both hypernymy and analogy tasks. The proposed model achieves comparable performance with the vanilla Word2Vec model on the cross-lingual analogy task, the hypernymy task shows that the cross-lingual Poincar\'e Word2Vec model can capture latent hierarchical structure from free text across languages, which are absent from the Euclidean-based Word2Vec representations. Our results show that by preserving the latent hierarchical information, hyperbolic spaces can offer better representations for cross-lingual embeddings.