Spectral Augmentations for Graph Contrastive Learning

TL;DR

This paper introduces spectral graph augmentation techniques for contrastive learning, enhancing the ability to capture structural properties across diverse graph datasets, leading to improved out-of-domain transfer performance.

Contribution

It proposes novel spectral-based graph augmentation methods derived from spectral analysis, providing effective transformations for contrastive learning of graph representations.

Findings

Spectral cropping improves structural feature learning.

Reordering frequency components enhances graph view diversity.

Out-of-domain transfer performance surpasses state-of-the-art methods.

Abstract

Contrastive learning has emerged as a premier method for learning representations with or without supervision. Recent studies have shown its utility in graph representation learning for pre-training. Despite successes, the understanding of how to design effective graph augmentations that can capture structural properties common to many different types of downstream graphs remains incomplete. We propose a set of well-motivated graph transformation operations derived via graph spectral analysis to provide a bank of candidates when constructing augmentations for a graph contrastive objective, enabling contrastive learning to capture useful structural representation from pre-training graph datasets. We first present a spectral graph cropping augmentation that involves filtering nodes by applying thresholds to the eigenvalues of the leading Laplacian eigenvectors. Our second novel augmentation reorders the graph frequency components in a structural Laplacian-derived position graph embedding. Further, we introduce a method that leads to improved views of local subgraphs by performing alignment via global random walk embeddings. Our experimental results indicate consistent improvements in out-of-domain graph data transfer compared to state-of-the-art graph contrastive learning methods, shedding light on how to design a graph learner that is able to learn structural properties common to diverse graph types.