Overlapping Spaces for Compact Graph Representations

TL;DR

This paper introduces overlapping spaces for graph embedding that automatically learn optimal configurations, are more compact, and outperform existing methods in preserving distances and rankings, with practical benefits in large-scale applications.

Contribution

The paper generalizes product spaces to overlapping spaces, eliminating the need for resource-intensive configuration search and enabling more compact, effective graph embeddings.

Findings

Overlapping spaces outperform competitors in graph embedding tasks.

They effectively preserve distances and rankings.

They reduce training time in large-scale applications.

Abstract

Various non-trivial spaces are becoming popular for embedding structured data such as graphs, texts, or images. Following spherical and hyperbolic spaces, more general product spaces have been proposed. However, searching for the best configuration of product space is a resource-intensive procedure, which reduces the practical applicability of the idea. We generalize the concept of product space and introduce an overlapping space that does not have the configuration search problem. The main idea is to allow subsets of coordinates to be shared between spaces of different types (Euclidean, hyperbolic, spherical). As a result, parameter optimization automatically learns the optimal configuration. Additionally, overlapping spaces allow for more compact representations since their geometry is more complex. Our experiments confirm that overlapping spaces outperform the competitors in graph embedding tasks. Here, we consider both distortion setup, where the aim is to preserve distances, and ranking setup, where the relative order should be preserved. The proposed method effectively solves the problem and outperforms the competitors in both settings. We also perform an empirical analysis in a realistic information retrieval task, where we compare all spaces by incorporating them into DSSM. In this case, the proposed overlapping space consistently achieves nearly optimal results without any configuration tuning. This allows for reducing training time, which can be significant in large-scale applications.