Unitary convolutions for learning on graphs and groups

TL;DR

This paper introduces unitary group convolutions for graph neural networks, improving stability and depth, and avoiding over-smoothing, leading to competitive performance on benchmarks.

Contribution

It proposes and analyzes unitary group convolutions, demonstrating their ability to enhance stability and depth in graph neural networks, and preventing over-smoothing.

Findings

Unitary graph convolutions avoid over-smoothing in GNNs.

Unitary GNNs achieve competitive benchmark performance.

Enhanced stability allows for deeper network architectures.

Abstract

Data with geometric structure is ubiquitous in machine learning often arising from fundamental symmetries in a domain, such as permutation-invariance in graphs and translation-invariance in images. Group-convolutional architectures, which encode symmetries as inductive bias, have shown great success in applications, but can suffer from instabilities as their depth increases and often struggle to learn long range dependencies in data. For instance, graph neural networks experience instability due to the convergence of node representations (over-smoothing), which can occur after only a few iterations of message-passing, reducing their effectiveness in downstream tasks. Here, we propose and study unitary group convolutions, which allow for deeper networks that are more stable during training. The main focus of the paper are graph neural networks, where we show that unitary graph convolutions provably avoid over-smoothing. Our experimental results confirm that unitary graph convolutional networks achieve competitive performance on benchmark datasets compared to state-of-the-art graph neural networks. We complement our analysis of the graph domain with the study of general unitary convolutions and analyze their role in enhancing stability in general group convolutional architectures.