Maximizing Cohesion and Separation in Graph Representation Learning: A Distance-aware Negative Sampling Approach

TL;DR

This paper introduces a Distance-aware Negative Sampling method for graph representation learning that improves the quality of node embeddings by balancing cohesion among close nodes and separation among distant nodes, enhancing downstream task performance.

Contribution

The paper proposes a novel negative sampling technique that considers pairwise shortest distances, improving graph embedding quality over traditional uniform sampling methods.

Findings

Enhanced node classification accuracy on benchmark datasets

Effective separation of distant node pairs in embeddings

Compatibility with various GRL algorithms

Abstract

The objective of unsupervised graph representation learning (GRL) is to learn a low-dimensional space of node embeddings that reflect the structure of a given unlabeled graph. Existing algorithms for this task rely on negative sampling objectives that maximize the similarity in node embeddings at nearby nodes (referred to as "cohesion") by maintaining positive and negative corpus of node pairs. While positive samples are drawn from node pairs that co-occur in short random walks, conventional approaches construct negative corpus by uniformly sampling random pairs, thus ignoring valuable information about structural dissimilarity among distant node pairs (referred to as "separation"). In this paper, we present a novel Distance-aware Negative Sampling (DNS) which maximizes the separation of distant node-pairs while maximizing cohesion at nearby node-pairs by setting the negative sampling probability proportional to the pair-wise shortest distances. Our approach can be used in conjunction with any GRL algorithm and we demonstrate the efficacy of our approach over baseline negative sampling methods over downstream node classification tasks on a number of benchmark datasets and GRL algorithms. All our codes and datasets are available at https://github.com/Distance-awareNS/DNS/.