Continuous Spiking Graph Neural Networks

TL;DR

This paper introduces Continuous Spiking Graph Neural Networks (COS-GNN), combining energy-efficient spiking neural networks with continuous graph neural dynamics to improve long-range dependency modeling and mitigate gradient issues.

Contribution

The paper proposes a novel unified framework integrating SNNs with CGNNs, utilizing high-order ODEs for better information preservation and gradient stability.

Findings

COS-GNN outperforms baseline models on graph learning tasks.

The approach effectively mitigates exploding and vanishing gradients.

Experimental results validate the energy efficiency and long-range dependency capture.

Abstract

Continuous graph neural networks (CGNNs) have garnered significant attention due to their ability to generalize existing discrete graph neural networks (GNNs) by introducing continuous dynamics. They typically draw inspiration from diffusion-based methods to introduce a novel propagation scheme, which is analyzed using ordinary differential equations (ODE). However, the implementation of CGNNs requires significant computational power, making them challenging to deploy on battery-powered devices. Inspired by recent spiking neural networks (SNNs), which emulate a biological inference process and provide an energy-efficient neural architecture, we incorporate the SNNs with CGNNs in a unified framework, named Continuous Spiking Graph Neural Networks (COS-GNN). We employ SNNs for graph node representation at each time step, which are further integrated into the ODE process along with time. To enhance information preservation and mitigate information loss in SNNs, we introduce the high-order structure of COS-GNN, which utilizes the second-order ODE for spiking representation and continuous propagation. Moreover, we provide the theoretical proof that COS-GNN effectively mitigates the issues of exploding and vanishing gradients, enabling us to capture long-range dependencies between nodes. Experimental results on graph-based learning tasks demonstrate the effectiveness of the proposed COS-GNN over competitive baselines.