Self-Supervised Node Representation Learning via Node-to-Neighbourhood Alignment

TL;DR

This paper introduces a novel self-supervised node representation learning method that aligns node and neighborhood representations through mutual information maximization, topology-aware sampling, and a negative-free decorrelation approach, achieving competitive classification results.

Contribution

The work presents a simple yet effective framework for self-supervised node embedding using node-to-neighborhood alignment, mutual information, and a negative-free decorrelation strategy, with theoretical and practical advancements.

Findings

Achieves promising node classification performance across various datasets.

Introduces a negative-free approach to avoid over-smoothing in contrastive learning.

Proposes a topology-aware positive sampling strategy for better structural alignment.

Abstract

Self-supervised node representation learning aims to learn node representations from unlabelled graphs that rival the supervised counterparts. The key towards learning informative node representations lies in how to effectively gain contextual information from the graph structure. In this work, we present simple-yet-effective self-supervised node representation learning via aligning the hidden representations of nodes and their neighbourhood. Our first idea achieves such node-to-neighbourhood alignment by directly maximizing the mutual information between their representations, which, we prove theoretically, plays the role of graph smoothing. Our framework is optimized via a surrogate contrastive loss and a Topology-Aware Positive Sampling (TAPS) strategy is proposed to sample positives by considering the structural dependencies between nodes, which enables offline positive selection. Considering the excessive memory overheads of contrastive learning, we further propose a negative-free solution, where the main contribution is a Graph Signal Decorrelation (GSD) constraint to avoid representation collapse and over-smoothing. The GSD constraint unifies some of the existing constraints and can be used to derive new implementations to combat representation collapse. By applying our methods on top of simple MLP-based node representation encoders, we learn node representations that achieve promising node classification performance on a set of graph-structured datasets from small- to large-scale.