Towards Understanding and Avoiding Limitations of Convolutions on Graphs

TL;DR

This paper provides a comprehensive theoretical analysis of message-passing neural networks (MPNNs), identifying key limitations like shared component amplification and component dominance, and proposes frameworks to mitigate these issues, enhancing their effectiveness.

Contribution

It introduces a detailed theoretical framework for understanding MPNNs' limitations and proposes new methods such as multi-relational split and spectral graph convolution to address these issues.

Findings

Identified shared component amplification (SCA) and component dominance (CD) as key limitations.

Proposed multi-relational split (MRS) framework to mitigate SCA.

Introduced MIMO-GC and LMGC for multi-channel spectral graph convolution.

Abstract

While message-passing neural networks (MPNNs) have shown promising results, their real-world impact remains limited. Although various limitations have been identified, their theoretical foundations remain poorly understood, leading to fragmented research efforts. In this thesis, we provide an in-depth theoretical analysis and identify several key properties limiting the performance of MPNNs. Building on these findings, we propose several frameworks that address these shortcomings. We identify two properties exhibited by many MPNNs: shared component amplification (SCA), where each message-passing iteration amplifies the same components across all feature channels, and component dominance (CD), where a single component gets increasingly amplified as more message-passing steps are applied. These properties lead to the observable phenomenon of rank collapse of node representations, which generalizes the established over-smoothing phenomenon. By generalizing and decomposing over-smoothing, we enable a deeper understanding of MPNNs, more targeted solutions, and more precise communication within the field. To avoid SCA, we show that utilizing multiple computational graphs or edge relations is necessary. Our multi-relational split (MRS) framework transforms any existing MPNN into one that leverages multiple edge relations. Additionally, we introduce the spectral graph convolution for multiple feature channels (MIMO-GC), which naturally uses multiple computational graphs. A localized variant, LMGC, approximates the MIMO-GC while inheriting its beneficial properties. To address CD, we demonstrate a close connection between MPNNs and the PageRank algorithm. Based on personalized PageRank, we propose a variant of MPNNs that allows for infinitely many message-passing iterations, while preserving initial node features. Collectively, these results deepen the theoretical understanding of MPNNs.