MUG: Meta-path-aware Universal Heterogeneous Graph Pre-Training

TL;DR

MUG introduces a novel pre-training framework for heterogeneous graphs that leverages meta-path awareness and a unified encoding strategy to improve transferability and generalization across diverse graph datasets.

Contribution

The paper proposes a meta-path-aware pre-training approach with a unified representation and shared encoder for heterogeneous graphs, addressing dataset heterogeneity challenges.

Findings

Effective transferability demonstrated on real datasets.

Outperforms existing methods in heterogeneous graph tasks.

Unified representation improves cross-dataset generalization.

Abstract

Universal graph pre-training has emerged as a key paradigm in graph representation learning, offering a promising way to train encoders to learn transferable representations from unlabeled graphs and to effectively generalize across a wide range of downstream tasks. However, recent explorations in universal graph pre-training primarily focus on homogeneous graphs and it remains unexplored for heterogeneous graphs, which exhibit greater structural and semantic complexity. This heterogeneity makes it highly challenging to train a universal encoder for diverse heterogeneous graphs: (i) the diverse types with dataset-specific semantics hinder the construction of a unified representation space; (ii) the number and semantics of meta-paths vary across datasets, making encoding and aggregation patterns learned from one dataset difficult to apply to others. To address these challenges, we propose a novel Meta-path-aware Universal heterogeneous Graph pre-training (MUG) approach. Specifically, for challenge (i), MUG introduces a input unification module that integrates information from multiple node and relation types within each heterogeneous graph into a unified representation.This representation is then projected into a shared space by a dimension-aware encoder, enabling alignment across graphs with diverse schemas.Furthermore, for challenge (ii), MUG trains a shared encoder to capture consistent structural patterns across diverse meta-path views rather than relying on dataset-specific aggregation strategies, while a global objective encourages discriminability and reduces dataset-specific biases. Extensive experiments demonstrate the effectiveness of MUG on some real datasets.