Scalable Heterogeneous Graph Learning via Heterogeneous-aware Orthogonal Prototype Experts

TL;DR

This paper introduces HOPE, a novel framework for heterogeneous graph neural networks that employs prototype-based routing and orthogonalization to improve prediction accuracy and diversity, addressing the limitations of traditional linear heads.

Contribution

HOPE provides a scalable, plug-and-play prediction head for HGNNs that effectively handles structural imbalance and long-tail distributions through prototype routing and expert orthogonalization.

Findings

HOPE achieves consistent SOTA improvements on four real datasets.

HOPE introduces a lightweight, flexible alternative to traditional linear heads.

Experiments show minimal overhead with significant performance gains.

Abstract

Heterogeneous Graph Neural Networks(HGNNs) have advanced mainly through better encoders, yet their decoding/projection stage still relies on a single shared linear head, assuming it can map rich node embeddings to labels. We call this the Linear Projection Bottleneck: in heterogeneous graphs, contextual diversity and long-tail shifts make a global head miss fine semantics, overfit hub nodes, and underserve tail nodes. While Mixture-of-Experts(MoE) could help, naively applying it clashes with structural imbalance and risks expert collapse. We propose a Heterogeneous-aware Orthogonal Prototype Experts framework named HOPE, a plug-and-play replacement for the standard prediction head. HOPE uses learnable prototype-based routing to assign instances to experts by similarity, letting expert usage follow the natural long-tail distribution, and adds expert orthogonalization to encourage diversity and prevent collapse. Experiments on four real datasets show consistent gains across SOTA HGNN backbones with minimal overhead.