Out-Of-Distribution Generalization on Graphs: A Survey

TL;DR

This survey comprehensively reviews recent advances in out-of-distribution generalization for graph machine learning, addressing the challenge of distribution shifts between training and testing data.

Contribution

It provides the first systematic review of OOD generalization on graphs, including formal definitions, categorization of methods, theoretical insights, and future directions.

Findings

Categorizes methods into data, model, and learning strategy classes

Summarizes theories related to OOD generalization on graphs

Reviews commonly used graph datasets for evaluation

Abstract

Graph machine learning has been extensively studied in both academia and industry. Although booming with a vast number of emerging methods and techniques, most of the literature is built on the in-distribution hypothesis, i.e., testing and training graph data are identically distributed. However, this in-distribution hypothesis can hardly be satisfied in many real-world graph scenarios where the model performance substantially degrades when there exist distribution shifts between testing and training graph data. To solve this critical problem, out-of-distribution (OOD) generalization on graphs, which goes beyond the in-distribution hypothesis, has made great progress and attracted ever-increasing attention from the research community. In this paper, we comprehensively survey OOD generalization on graphs and present a detailed review of recent advances in this area. First, we provide a formal problem definition of OOD generalization on graphs. Second, we categorize existing methods into three classes from conceptually different perspectives, i.e., data, model, and learning strategy, based on their positions in the graph machine learning pipeline, followed by detailed discussions for each category. We also review the theories related to OOD generalization on graphs and introduce the commonly used graph datasets for thorough evaluations. Finally, we share our insights on future research directions. This paper is the first systematic and comprehensive review of OOD generalization on graphs, to the best of our knowledge.