DuSEGO: Dual Second-order Equivariant Graph Ordinary Differential Equation

TL;DR

DuSEGO introduces a dual second-order equivariant graph ODE framework that enhances GNN expressiveness, mitigates over-smoothing, and stabilizes training in deep models by leveraging second-order dynamics.

Contribution

The paper proposes DuSEGO, a novel dual second-order equivariant graph ODE model that maintains equivariance and addresses over-smoothing and gradient issues in deep GNNs.

Findings

Outperforms baseline models on benchmark datasets.

Effectively alleviates over-smoothing in feature and coordinate updates.

Mitigates gradient explosion and vanishing problems in deep GNNs.

Abstract

Graph Neural Networks (GNNs) with equivariant properties have achieved significant success in modeling complex dynamic systems and molecular properties. However, their expressiveness ability is limited by: (1) Existing methods often overlook the over-smoothing issue caused by traditional GNN models, as well as the gradient explosion or vanishing problems in deep GNNs. (2) Most models operate on first-order information, neglecting that the real world often consists of second-order systems, which further limits the model's representation capabilities. To address these issues, we propose the \textbf{Du}al \textbf{S}econd-order \textbf{E}quivariant \textbf{G}raph \textbf{O}rdinary Differential Equation (\method{}) for equivariant representation. Specifically, \method{} apply the dual second-order equivariant graph ordinary differential equations (Graph ODEs) on graph embeddings and node coordinates, simultaneously. Theoretically, we first prove that \method{} maintains the equivariant property. Furthermore, we provide theoretical insights showing that \method{} effectively alleviates the over-smoothing problem in both feature representation and coordinate update. Additionally, we demonstrate that the proposed \method{} mitigates the exploding and vanishing gradients problem, facilitating the training of deep multi-layer GNNs. Extensive experiments on benchmark datasets validate the superiority of the proposed \method{} compared to baselines.