Foundations and Frontiers of Graph Learning Theory

TL;DR

This paper reviews the theoretical foundations of graph learning, focusing on GNNs, their expressive power, generalization, optimization, and phenomena like over-smoothing, providing insights into core principles and future challenges.

Contribution

It offers a comprehensive summary of recent theoretical advances in graph learning, highlighting key principles and identifying open challenges for future research.

Findings

Analysis of GNN expressiveness and limitations

Insights into over-smoothing and over-squashing phenomena

Discussion of theoretical challenges and potential solutions

Abstract

Recent advancements in graph learning have revolutionized the way to understand and analyze data with complex structures. Notably, Graph Neural Networks (GNNs), i.e. neural network architectures designed for learning graph representations, have become a popular paradigm. With these models being usually characterized by intuition-driven design or highly intricate components, placing them within the theoretical analysis framework to distill the core concepts, helps understand the key principles that drive the functionality better and guide further development. Given this surge in interest, this article provides a comprehensive summary of the theoretical foundations and breakthroughs concerning the approximation and learning behaviors intrinsic to prevalent graph learning models. Encompassing discussions on fundamental aspects such as expressiveness power, generalization, optimization, and unique phenomena such as over-smoothing and over-squashing, this piece delves into the theoretical foundations and frontier driving the evolution of graph learning. In addition, this article also presents several challenges and further initiates discussions on possible solutions.