Node Representation Learning in Graph via Node-to-Neighbourhood Mutual Information Maximization

TL;DR

This paper introduces a self-supervised node representation learning method that maximizes mutual information between nodes and their neighborhoods, improving efficiency and performance in graph-based tasks.

Contribution

The work proposes a mutual information maximization framework with a topology-aware positive sampling strategy, enabling faster and more effective node embedding learning.

Findings

Achieves promising results on node classification datasets.

Orders of magnitude faster than existing methods when using MLP encoders.

Theoretically justified by its link to graph smoothing.

Abstract

The key towards learning informative node representations in graphs lies in how to gain contextual information from the neighbourhood. In this work, we present a simple-yet-effective self-supervised node representation learning strategy via directly maximizing the mutual information between the hidden representations of nodes and their neighbourhood, which can be theoretically justified by its link to graph smoothing. Following InfoNCE, our framework is optimized via a surrogate contrastive loss, where the positive selection underpins the quality and efficiency of representation learning. To this end, we propose a topology-aware positive sampling strategy, which samples positives from the neighbourhood by considering the structural dependencies between nodes and thus enables positive selection upfront. In the extreme case when only one positive is sampled, we fully avoid expensive neighbourhood aggregation. Our methods achieve promising performance on various node classification datasets. It is also worth mentioning by applying our loss function to MLP based node encoders, our methods can be orders of faster than existing solutions. Our codes and supplementary materials are available at https://github.com/dongwei156/n2n.