Exploring the Semantic Content of Unsupervised Graph Embeddings: An Empirical Study

TL;DR

This study investigates whether unsupervised graph embeddings encode topological features of graphs by predicting these features from embeddings, providing insights into the structural information captured by embedding methods.

Contribution

It demonstrates that several topological features can be approximated from embeddings, offering a theoretical understanding of what structural information embeddings encode.

Findings

Topological features are partially captured in embedding spaces

Supervised and unsupervised methods can predict features from embeddings

Different embedding techniques vary in how well they encode topological information

Abstract

Graph embeddings have become a key and widely used technique within the field of graph mining, proving to be successful across a broad range of domains including social, citation, transportation and biological. Graph embedding techniques aim to automatically create a low-dimensional representation of a given graph, which captures key structural elements in the resulting embedding space. However, to date, there has been little work exploring exactly which topological structures are being learned in the embeddings process. In this paper, we investigate if graph embeddings are approximating something analogous with traditional vertex level graph features. If such a relationship can be found, it could be used to provide a theoretical insight into how graph embedding approaches function. We perform this investigation by predicting known topological features, using supervised and unsupervised methods, directly from the embedding space. If a mapping between the embeddings and topological features can be found, then we argue that the structural information encapsulated by the features is represented in the embedding space. To explore this, we present extensive experimental evaluation from five state-of-the-art unsupervised graph embedding techniques, across a range of empirical graph datasets, measuring a selection of topological features. We demonstrate that several topological features are indeed being approximated by the embedding space, allowing key insight into how graph embeddings create good representations.