HeteGraph-Mamba: Heterogeneous Graph Learning via Selective State Space Model

TL;DR

HeteGraph-Mamba introduces a novel heterogeneous graph learning model using selective state space models, effectively capturing long-range dependencies and outperforming existing methods in accuracy and efficiency.

Contribution

The paper presents the first heterogeneous graph learning model leveraging SSSMs, with a two-level tokenization approach for improved dependency capturing and efficiency.

Findings

Outperforms 19 state-of-the-art methods in accuracy.

Demonstrates superior efficiency in heterogeneous graph tasks.

Effectively captures long-range dependencies among diverse nodes.

Abstract

We propose a heterogeneous graph mamba network (HGMN) as the first exploration in leveraging the selective state space models (SSSMs) for heterogeneous graph learning. Compared with the literature, our HGMN overcomes two major challenges: (i) capturing long-range dependencies among heterogeneous nodes and (ii) adapting SSSMs to heterogeneous graph data. Our key contribution is a general graph architecture that can solve heterogeneous nodes in real-world scenarios, followed an efficient flow. Methodologically, we introduce a two-level efficient tokenization approach that first captures long-range dependencies within identical node types, and subsequently across all node types. Empirically, we conduct comparisons between our framework and 19 state-of-the-art methods on the heterogeneous benchmarks. The extensive comparisons demonstrate that our framework outperforms other methods in both the accuracy and efficiency dimensions.