Descent Steps of a Relation-Aware Energy Produce Heterogeneous Graph Neural Networks

TL;DR

This paper introduces a relation-aware energy-based heterogeneous GNN architecture that balances oversmoothing and long-range dependency capturing, utilizing bilevel optimization for improved node classification across multiple benchmarks.

Contribution

It proposes a novel energy function-based heterogeneous GNN with bilevel optimization, effectively modeling diverse heterophily and mitigating oversmoothing in deep networks.

Findings

Achieves competitive accuracy on 8 heterogeneous graph benchmarks.

Effectively models heterophily relationships between node types.

Balances oversmoothing and long-range dependency capture.

Abstract

Heterogeneous graph neural networks (GNNs) achieve strong performance on node classification tasks in a semi-supervised learning setting. However, as in the simpler homogeneous GNN case, message-passing-based heterogeneous GNNs may struggle to balance between resisting the oversmoothing that may occur in deep models, and capturing long-range dependencies of graph structured data. Moreover, the complexity of this trade-off is compounded in the heterogeneous graph case due to the disparate heterophily relationships between nodes of different types. To address these issues, we propose a novel heterogeneous GNN architecture in which layers are derived from optimization steps that descend a novel relation-aware energy function. The corresponding minimizer is fully differentiable with respect to the energy function parameters, such that bilevel optimization can be applied to effectively learn a functional form whose minimum provides optimal node representations for subsequent classification tasks. In particular, this methodology allows us to model diverse heterophily relationships between different node types while avoiding oversmoothing effects. Experimental results on 8 heterogeneous graph benchmarks demonstrates that our proposed method can achieve competitive node classification accuracy