DeepWalking Backwards: From Embeddings Back to Graphs

TL;DR

This paper investigates whether low-dimensional graph embeddings can be inverted to recover the original graph, revealing what structural information is preserved or lost in the embedding process.

Contribution

It introduces algorithms for approximately inverting DeepWalk embeddings to recover original graphs and analyzes the extent of information retention.

Findings

Community structure is often preserved or enhanced in the inverted graph.

Significant edge and property information like triangle density is often lost.

Embedding inversion can produce graphs with similar embeddings to the original.

Abstract

Low-dimensional node embeddings play a key role in analyzing graph datasets. However, little work studies exactly what information is encoded by popular embedding methods, and how this information correlates with performance in downstream machine learning tasks. We tackle this question by studying whether embeddings can be inverted to (approximately) recover the graph used to generate them. Focusing on a variant of the popular DeepWalk method (Perozzi et al., 2014; Qiu et al., 2018), we present algorithms for accurate embedding inversion - i.e., from the low-dimensional embedding of a graph G, we can find a graph H with a very similar embedding. We perform numerous experiments on real-world networks, observing that significant information about G, such as specific edges and bulk properties like triangle density, is often lost in H. However, community structure is often preserved or even enhanced. Our findings are a step towards a more rigorous understanding of exactly what information embeddings encode about the input graph, and why this information is useful for learning tasks.