Mixture of Scope Experts at Test: Generalizing Deeper Graph Neural Networks with Shallow Variants

TL;DR

This paper introduces Moscat, a method that combines shallow and deep GNNs at test time to improve performance on heterophilous graphs, addressing depth-related generalization issues.

Contribution

It presents a novel mixture of scope experts approach that enhances deep GNN generalization without sacrificing expressivity, supported by theoretical and empirical analysis.

Findings

Moscat significantly improves GNN accuracy across datasets.

It effectively balances shallow and deep GNN advantages.

The method is compatible with various GNN architectures.

Abstract

Heterophilous graphs, where dissimilar nodes tend to connect, pose a challenge for graph neural networks (GNNs). Increasing the GNN depth can expand the scope (i.e., receptive field), potentially finding homophily from the higher-order neighborhoods. However, GNNs suffer from performance degradation as depth increases. Despite having better expressivity, state-of-the-art deeper GNNs achieve only marginal improvements compared to their shallow variants. Through theoretical and empirical analysis, we systematically demonstrate a shift in GNN generalization preferences across nodes with different homophily levels as depth increases. This creates a disparity in generalization patterns between GNN models with varying depth. Based on these findings, we propose to improve deeper GNN generalization while maintaining high expressivity by Mixture of scope experts at test (Moscat). Experimental results show that Moscat works flexibly with various GNNs across a wide range of datasets while significantly improving accuracy. Our code is available at (https://github.com/Hydrapse/moscat).