Echoless Label-Based Pre-computation for Memory-Efficient Heterogeneous Graph Learning

TL;DR

This paper introduces Echoless-LP, a memory-efficient pre-computation method for heterogeneous graph neural networks that prevents label leakage during training, improving performance on large-scale graphs.

Contribution

It proposes a novel echoless propagation technique with partitioning and adjustment mechanisms to eliminate label leakage while maintaining efficiency.

Findings

Echoless-LP outperforms baseline methods in accuracy.

Echoless-LP maintains memory efficiency on large graphs.

The method is compatible with various message passing techniques.

Abstract

Heterogeneous Graph Neural Networks (HGNNs) are widely used for deep learning on heterogeneous graphs. Typical end-to-end HGNNs require repetitive message passing during training, limiting efficiency for large-scale real-world graphs. Pre-computation-based HGNNs address this by performing message passing only once during preprocessing, collecting neighbor information into regular-shaped tensors, which enables efficient mini-batch training. Label-based pre-computation methods collect neighbors' label information but suffer from training label leakage, where a node's own label information propagates back to itself during multi-hop message passing - the echo effect. Existing mitigation strategies are memory-inefficient on large graphs or suffer from compatibility issues with advanced message passing methods. We propose Echoless Label-based Pre-computation (Echoless-LP), which eliminates training label leakage with Partition-Focused Echoless Propagation (PFEP). PFEP partitions target nodes and performs echoless propagation, where nodes in each partition collect label information only from neighbors in other partitions, avoiding echo while remaining memory-efficient and compatible with any message passing method. We also introduce an Asymmetric Partitioning Scheme (APS) and a PostAdjust mechanism to address information loss from partitioning and distributional shifts across partitions. Experiments on public datasets demonstrate that Echoless-LP achieves superior performance and maintains memory efficiency compared to baselines.