Learning Representations using Spectral-Biased Random Walks on Graphs

TL;DR

This paper introduces a spectral-bias in random walks for graph embedding, leveraging neighborhood spectral measures and Wasserstein regularization to enhance node representation quality.

Contribution

It proposes a novel spectral-biased random walk model with Wasserstein regularization, improving node embeddings over existing methods.

Findings

Significant improvements in link prediction accuracy.

Enhanced node classification performance.

Effective neighborhood spectral measure utilization.

Abstract

Several state-of-the-art neural graph embedding methods are based on short random walks (stochastic processes) because of their ease of computation, simplicity in capturing complex local graph properties, scalability, and interpretibility. In this work, we are interested in studying how much a probabilistic bias in this stochastic process affects the quality of the nodes picked by the process. In particular, our biased walk, with a certain probability, favors movement towards nodes whose neighborhoods bear a structural resemblance to the current node's neighborhood. We succinctly capture this neighborhood as a probability measure based on the spectrum of the node's neighborhood subgraph represented as a normalized laplacian matrix. We propose the use of a paragraph vector model with a novel Wasserstein regularization term. We empirically evaluate our approach against several state-of-the-art node embedding techniques on a wide variety of real-world datasets and demonstrate that our proposed method significantly improves upon existing methods on both link prediction and node classification tasks.