From Hyperbolic Geometry Back to Word Embeddings

TL;DR

This paper explores the use of hyperbolic geometry for word embeddings, proposing that random points in hyperbolic space can serve as initial representations, and discusses methods to align these with actual words.

Contribution

It introduces a novel perspective of using hyperbolic space for word embeddings and proposes alignment techniques to map points to specific words.

Findings

Hyperbolic points can serve as initial word representations.

Alignment techniques can match points to words using PMI.

Potential for improved semantic representations.

Abstract

We choose random points in the hyperbolic disc and claim that these points are already word representations. However, it is yet to be uncovered which point corresponds to which word of the human language of interest. This correspondence can be approximately established using a pointwise mutual information between words and recent alignment techniques.