Rethinking Message Passing Neural Networks with Diffusion Distance-guided Stress Majorization

TL;DR

This paper introduces DDSM, a novel message passing neural network that uses diffusion distance-guided stress majorization to address over-smoothing and over-correlation issues, achieving superior performance on various graph types.

Contribution

The paper presents a new MPNN model incorporating stress majorization and diffusion distances, with theoretical analysis and efficient algorithms, improving over existing models.

Findings

DDSM outperforms 15 strong baselines on multiple graph datasets.

The model effectively mitigates over-smoothing and over-correlation.

Theoretical guarantees support the proposed algorithms.

Abstract

Message passing neural networks (MPNNs) have emerged as go-to models for learning on graph-structured data in the past decade. Despite their effectiveness, most of such models still incur severe issues such as over-smoothing and -correlation, due to their underlying objective of minimizing the Dirichlet energy and the derived neighborhood aggregation operations. In this paper, we propose the DDSM, a new MPNN model built on an optimization framework that includes the stress majorization and orthogonal regularization for overcoming the above issues. Further, we introduce the diffusion distances for nodes into the framework to guide the new message passing operations and develop efficient algorithms for distance approximations, both backed by rigorous theoretical analyses. Our comprehensive experiments showcase that DDSM consistently and considerably outperforms 15 strong baselines on both homophilic and heterophilic graphs.