Mixture of Message Passing Experts with Routing Entropy Regularization for Node Classification

TL;DR

GNNMoE introduces a mixture of message passing experts with entropy regularization for adaptive, fine-grained node classification, especially effective in heterophilous graph structures.

Contribution

It proposes a novel entropy-driven mixture of experts framework with hybrid routing and regularization for improved node representation learning.

Findings

Outperforms state-of-the-art methods on 12 benchmark datasets.

Maintains scalability and interpretability.

Adapts to diverse neighborhood contexts effectively.

Abstract

Graph neural networks (GNNs) have achieved significant progress in graph-based learning tasks, yet their performance often deteriorates when facing heterophilous structures where connected nodes differ substantially in features and labels. To address this limitation, we propose GNNMoE, a novel entropy-driven mixture of message-passing experts framework that enables node-level adaptive representation learning. GNNMoE decomposes message passing into propagation and transformation operations and integrates them through multiple expert networks guided by a hybrid routing mechanism. And a routing entropy regularization dynamically adjusts soft weighting and soft top-$k$ routing, allowing GNNMoE to flexibly adapt to diverse neighborhood contexts. Extensive experiments on twelve benchmark datasets demonstrate that GNNMoE consistently outperforms SOTA node classification methods, while maintaining scalability and interpretability. This work provides a unified and principled approach for achieving fine-grained, personalized node representation learning.