Graph Neural Networks Need Cluster-Normalize-Activate Modules

TL;DR

This paper introduces a Cluster-Normalize-Activate (CNA) module for Graph Neural Networks that mitigates oversmoothing, enhances accuracy in node classification and property prediction, and reduces model complexity.

Contribution

The paper proposes a novel plug-and-play CNA module that enables GNNs to form super nodes, improving performance and efficiency across various tasks.

Findings

CNA improves accuracy on Cora and CiteSeer datasets.

GNNs with CNA require fewer parameters than existing models.

CNA benefits regression tasks by reducing mean squared error.

Abstract

Graph Neural Networks (GNNs) are non-Euclidean deep learning models for graph-structured data. Despite their successful and diverse applications, oversmoothing prohibits deep architectures due to node features converging to a single fixed point. This severely limits their potential to solve complex tasks. To counteract this tendency, we propose a plug-and-play module consisting of three steps: Cluster-Normalize-Activate (CNA). By applying CNA modules, GNNs search and form super nodes in each layer, which are normalized and activated individually. We demonstrate in node classification and property prediction tasks that CNA significantly improves the accuracy over the state-of-the-art. Particularly, CNA reaches 94.18% and 95.75% accuracy on Cora and CiteSeer, respectively. It further benefits GNNs in regression tasks as well, reducing the mean squared error compared to all baselines. At the same time, GNNs with CNA require substantially fewer learnable parameters than competing architectures.