Self-Supervised Learning of Graph Neural Networks: A Unified Review

TL;DR

This paper provides a comprehensive review of self-supervised learning methods for graph neural networks, categorizing approaches, unifying frameworks, and establishing a standardized testbed for evaluation.

Contribution

It offers a unified framework for SSL methods in GNNs, categorizes existing approaches, and develops a standardized testbed for fair comparison and further research.

Findings

Unified framework for contrastive and predictive SSL methods for GNNs

Categorization of SSL methods and datasets in GNNs

Development of a standardized testbed for evaluation

Abstract

Deep models trained in supervised mode have achieved remarkable success on a variety of tasks. When labeled samples are limited, self-supervised learning (SSL) is emerging as a new paradigm for making use of large amounts of unlabeled samples. SSL has achieved promising performance on natural language and image learning tasks. Recently, there is a trend to extend such success to graph data using graph neural networks (GNNs). In this survey, we provide a unified review of different ways of training GNNs using SSL. Specifically, we categorize SSL methods into contrastive and predictive models. In either category, we provide a unified framework for methods as well as how these methods differ in each component under the framework. Our unified treatment of SSL methods for GNNs sheds light on the similarities and differences of various methods, setting the stage for developing new methods and algorithms. We also summarize different SSL settings and the corresponding datasets used in each setting. To facilitate methodological development and empirical comparison, we develop a standardized testbed for SSL in GNNs, including implementations of common baseline methods, datasets, and evaluation metrics.