A Broader Picture of Random-walk Based Graph Embedding

TL;DR

This paper presents an analytical framework for random-walk based graph embedding, categorizing existing methods, proposing new approaches, and demonstrating significant improvements in link prediction performance.

Contribution

It introduces a comprehensive framework that explains, categorizes, and extends random-walk based graph embedding methods, highlighting new multi-scale embedding strategies.

Findings

Autocovariance similarity with dot product ranking outperforms PMI-based methods by up to 100%.

The framework categorizes many existing approaches and suggests new ones.

Multi-scale embeddings improve downstream task performance.

Abstract

Graph embedding based on random-walks supports effective solutions for many graph-related downstream tasks. However, the abundance of embedding literature has made it increasingly difficult to compare existing methods and to identify opportunities to advance the state-of-the-art. Meanwhile, existing work has left several fundamental questions -- such as how embeddings capture different structural scales and how they should be applied for effective link prediction -- unanswered. This paper addresses these challenges with an analytical framework for random-walk based graph embedding that consists of three components: a random-walk process, a similarity function, and an embedding algorithm. Our framework not only categorizes many existing approaches but naturally motivates new ones. With it, we illustrate novel ways to incorporate embeddings at multiple scales to improve downstream task performance. We also show that embeddings based on autocovariance similarity, when paired with dot product ranking for link prediction, outperform state-of-the-art methods based on Pointwise Mutual Information similarity by up to 100%.