Reversible and irreversible bracket-based dynamics for deep graph neural networks

TL;DR

This paper introduces novel graph neural network architectures based on bracket-based dynamical systems, providing a unified, explainable framework for reversible and irreversible deep GNNs that preserve or dissipate energy.

Contribution

It proposes a new class of GNNs grounded in structure-preserving dynamical systems, clarifying the roles of reversibility and irreversibility in network performance.

Findings

The proposed architectures are provably energy-conserving or dissipative.

The framework explains deviations in existing models from theoretical principles.

Results highlight the importance of energy dynamics in deep GNNs.

Abstract

Recent works have shown that physics-inspired architectures allow the training of deep graph neural networks (GNNs) without oversmoothing. The role of these physics is unclear, however, with successful examples of both reversible (e.g., Hamiltonian) and irreversible (e.g., diffusion) phenomena producing comparable results despite diametrically opposed mechanisms, and further complications arising due to empirical departures from mathematical theory. This work presents a series of novel GNN architectures based upon structure-preserving bracket-based dynamical systems, which are provably guaranteed to either conserve energy or generate positive dissipation with increasing depth. It is shown that the theoretically principled framework employed here allows for inherently explainable constructions, which contextualize departures from theory in current architectures and better elucidate the roles of reversibility and irreversibility in network performance.