Language Models as Messengers: Enhancing Message Passing in Heterophilic Graph Learning

TL;DR

This paper introduces LEMP4HG, a language model-based message passing method that improves heterophilic graph learning by leveraging node text semantics and an active learning strategy, outperforming existing methods.

Contribution

The paper proposes a novel LM-enhanced message passing approach for heterophilic graphs, explicitly modeling semantic relationships from node texts and employing an active learning strategy for efficiency.

Findings

LEMP4HG outperforms state-of-the-art methods on heterophilic graphs.

It maintains strong performance on homophilic graphs.

The approach is computationally efficient with practical budgets.

Abstract

Graph neural networks (GNNs) have become a standard paradigm for graph representation learning, yet their message passing mechanism implicitly assumes that messages can be represented by source node embeddings, an assumption that fails in heterophilic graphs. While existing methods attempt to address heterophily through graph structure refinement or adaptation of neighbor aggregation, they often overlook the semantic potential of node text, relying on suboptimal message representation for propagation and compromise performance on homophilic graphs. To address these limitations, we propose LEMP4HG, a novel language model (LM)-enhanced message passing approach for heterophilic graph learning. Specifically, for text-attributed graphs (TAG), we leverage a LM to explicitly model inter-node semantic relationships from paired node texts, synthesizing semantically informed messages for propagation. To ensure practical efficiency, we further introduce an active learning-inspired strategy guided by a tailored heuristic, MVRD, which selectively enhances messages for node pairs most affected by message passing. Extensive experiments demonstrate that LEMP4HG consistently outperforms state-of-the-art methods on heterophilic graphs while maintaining robust performance on homophilic graphs under a practical computational budget.