Cross-attentive Cohesive Subgraph Embedding to Mitigate Oversquashing in GNNs

TL;DR

This paper introduces a cross-attentive cohesive subgraph embedding method to reduce oversquashing in GNNs, improving global context capture and classification accuracy.

Contribution

It presents a novel framework that enhances node embeddings by emphasizing cohesive long-range structures, addressing oversquashing in dense and heterophilic graphs.

Findings

Achieves consistent accuracy improvements on benchmark datasets.

Effectively mitigates oversquashing by emphasizing relevant long-range structures.

Enhances global context capture without overloading message-passing channels.

Abstract

Graph neural networks (GNNs) have achieved strong performance across various real-world domains. Nevertheless, they suffer from oversquashing, where long-range information is distorted as it is compressed through limited message-passing pathways. This bottleneck limits their ability to capture essential global context and decreases their performance, particularly in dense and heterophilic regions of graphs. To address this issue, we propose a novel graph learning framework that enriches node embeddings via cross-attentive cohesive subgraph representations to mitigate the impact of excessive long-range dependencies. This framework enhances the node representation by emphasizing cohesive structure in long-range information but removing noisy or irrelevant connections. It preserves essential global context without overloading the narrow bottlenecked channels, which further mitigates oversquashing. Extensive experiments on multiple benchmark datasets demonstrate that our model achieves consistent improvements in classification accuracy over standard baseline methods.