Energy Guided smoothness to improve Robustness in Graph Classification

TL;DR

This paper investigates how the smoothness of node representations in Graph Neural Networks (GNNs), measured by Dirichlet Energy, influences robustness to label noise, and proposes training strategies to enhance this robustness without sacrificing performance.

Contribution

It introduces novel training methods that incorporate smoothness bias via Dirichlet Energy minimization, improving GNN robustness to noisy labels.

Findings

Reducing Dirichlet Energy improves GNN robustness.

Proposed strategies do not harm noise-free performance.

Smoothness bias enhances generalization under label noise.

Abstract

Graph Neural Networks (GNNs) are powerful at solving graph classification tasks, yet applied problems often contain noisy labels. In this work, we study GNN robustness to label noise, demonstrate GNN failure modes when models struggle to generalise on low-order graphs, low label coverage, or when a model is over-parameterized. We establish both empirical and theoretical links between GNN robustness and the reduction of the total Dirichlet Energy of learned node representations, which encapsulates the hypothesized GNN smoothness inductive bias. Finally, we introduce two training strategies to enhance GNN robustness: (1) by incorporating a novel inductive bias in the weight matrices through the removal of negative eigenvalues, connected to Dirichlet Energy minimization; (2) by extending to GNNs a loss penalty that promotes learned smoothness. Importantly, neither approach negatively impacts performance in noise-free settings, supporting our hypothesis that the source of GNNs robustness is their smoothness inductive bias.