Learning Graph Node Embeddings by Smooth Pair Sampling

TL;DR

This paper introduces a novel regularization technique for node embedding that uses smoothed pair sampling to address skewed pair frequency distributions, improving learning efficiency and effectiveness.

Contribution

It proposes an efficient sampling method based on smoothed pair frequencies to enhance random walk-based node embedding models.

Findings

The new sampling method improves embedding quality.

The approach reduces dominance of frequent pairs in learning.

Theoretical analysis supports the effectiveness of smoothing.

Abstract

Random walk-based node embedding algorithms have attracted a lot of attention due to their scalability and ease of implementation. Previous research has focused on different walk strategies, optimization objectives, and embedding learning models. Inspired by observations on real data, we take a different approach and propose a new regularization technique. More precisely, the frequencies of node pairs generated by the skip-gram model on random walk node sequences follow a highly skewed distribution which causes learning to be dominated by a fraction of the pairs. We address the issue by designing an efficient sampling procedure that generates node pairs according to their {\em smoothed frequency}. Theoretical and experimental results demonstrate the advantages of our approach.