All Nodes are created Not Equal: Node-Specific Layer Aggregation and Filtration for GNN

TL;DR

This paper introduces NoSAF, a novel GNN framework that employs node-specific layer aggregation and filtration, effectively addressing over-smoothing, heterophily, and enabling deeper networks with improved robustness.

Contribution

We propose NoSAF, a framework with node-specific filtering and dynamic codebank optimization, overcoming over-smoothing and heterophily in deep GNNs.

Findings

NoSAF effectively deepens GNNs beyond traditional limits.

It overcomes heterophily issues in GNNs.

NoSAF-D enhances information retention in deep layers.

Abstract

The ever-designed Graph Neural Networks, though opening a promising path for the modeling of the graph-structure data, unfortunately introduce two daunting obstacles to their deployment on devices. (I) Most of existing GNNs are shallow, due mostly to the over-smoothing and gradient-vanish problem as they go deeper as convolutional architectures. (II) The vast majority of GNNs adhere to the homophily assumption, where the central node and its adjacent nodes share the same label. This assumption often poses challenges for many GNNs working with heterophilic graphs. Addressing the aforementioned issue has become a looming challenge in enhancing the robustness and scalability of GNN applications. In this paper, we take a comprehensive and systematic approach to overcoming the two aforementioned challenges for the first time. We propose a Node-Specific Layer Aggregation and Filtration architecture, termed NoSAF, a framework capable of filtering and processing information from each individual nodes. NoSAF introduces the concept of "All Nodes are Created Not Equal" into every layer of deep networks, aiming to provide a reliable information filter for each layer's nodes to sieve out information beneficial for the subsequent layer. By incorporating a dynamically updated codebank, NoSAF dynamically optimizes the optimal information outputted downwards at each layer. This effectively overcomes heterophilic issues and aids in deepening the network. To compensate for the information loss caused by the continuous filtering in NoSAF, we also propose NoSAF-D (Deep), which incorporates a compensation mechanism that replenishes information in every layer of the model, allowing NoSAF to perform meaningful computations even in very deep layers.