The Self-Loop Paradox: Investigating the Impact of Self-Loops on Graph Neural Networks

TL;DR

This paper reveals a paradoxical effect in graph neural networks where adding self-loops can reduce the self-information flow in multi-layer GNNs, depending on network depth and architecture.

Contribution

It introduces the self-loop paradox, providing an analytical framework to understand how self-loops influence information flow in GNNs, supported by theoretical and experimental validation.

Findings

Self-loops can decrease self-information flow in certain GNN architectures.

The paradox depends on the number of layers and whether the layer count is even or odd.

Experimental validation confirms the theoretical predictions on real-world graphs.

Abstract

Many Graph Neural Networks (GNNs) add self-loops to a graph to include feature information about a node itself at each layer. However, if the GNN consists of more than one layer, this information can return to its origin via cycles in the graph topology. Intuition suggests that this "backflow" of information should be larger in graphs with self-loops compared to graphs without. In this work, we counter this intuition and show that for certain GNN architectures, the information a node gains from itself can be smaller in graphs with self-loops compared to the same graphs without. We adopt an analytical approach for the study of statistical graph ensembles with a given degree sequence and show that this phenomenon, which we call the self-loop paradox, can depend both on the number of GNN layers $k$ and whether $k$ is even or odd. We experimentally validate our theoretical findings in a synthetic node classification task and investigate its practical relevance in 23 real-world graphs.