Towards Invariance to Node Identifiers in Graph Neural Networks

TL;DR

This paper addresses the problem of node ID dependence in Graph Neural Networks, proposing a novel regularization method to enforce ID invariance, which improves generalization and expressivity.

Contribution

It identifies the limitations of ID-based GNNs and introduces a new regularization technique to ensure ID invariance, enhancing model robustness.

Findings

The proposed regularization improves ID invariance in GNNs.

Enforcing ID invariance boosts generalization performance.

The approach is effective on both real-world and synthetic tasks.

Abstract

Message-Passing Graph Neural Networks (GNNs) are known to have limited expressive power, due to their message passing structure. One mechanism for circumventing this limitation is to add unique node identifiers (IDs), which break the symmetries that underlie the expressivity limitation. In this work, we highlight a key limitation of the ID framework, and propose an approach for addressing it. We begin by observing that the final output of the GNN should clearly not depend on the specific IDs used. We then show that in practice this does not hold, and thus the learned network does not possess this desired structural property. Such invariance to node IDs may be enforced in several ways, and we discuss their theoretical properties. We then propose a novel regularization method that effectively enforces ID invariance to the network. Extensive evaluations on both real-world and synthetic tasks demonstrate that our approach significantly improves ID invariance and, in turn, often boosts generalization performance.