Aligning the Spectrum: Hybrid Graph Pre-training and Prompt Tuning across Homophily and Heterophily

TL;DR

This paper introduces a spectral alignment approach for graph pre-training and prompt tuning, addressing spectral diversity in real-world graphs to improve knowledge transfer across different graph types.

Contribution

It proposes a hybrid spectral backbone and spectral-aligned prompt tuning to overcome limitations of single-filter models, enabling better spectral matching and knowledge utilization.

Findings

Effective in both homophily and heterophily graphs

Improves knowledge transfer in limited supervision scenarios

Validates through extensive experiments in various settings

Abstract

Graph ``pre-training and prompt-tuning'' aligns downstream tasks with pre-trained objectives to enable efficient knowledge transfer under limited supervision. However, current methods typically rely on single-filter backbones (e.g., low-pass), whereas real-world graphs exhibit inherent spectral diversity. Our theoretical \textit{Spectral Specificity} principle reveals that effective knowledge transfer requires alignment between pre-trained spectral filters and the intrinsic spectrum of downstream graphs. This identifies two fundamental limitations: (1) Knowledge Bottleneck: single-filter models suffer from irreversible information loss by suppressing signals from other frequency bands (e.g., high-frequency); (2) Utilization Bottleneck: spectral mismatches between pre-trained filters and downstream spectra lead to significant underutilization of pre-trained knowledge. To bridge this gap, we propose HS-GPPT. We utilize a hybrid spectral backbone to construct an abundant knowledge basis. Crucially, we introduce Spectral-Aligned Prompt Tuning to actively align the downstream graph's spectrum with diverse pre-trained filters, facilitating comprehensive knowledge utilization across both homophily and heterophily. Extensive experiments validate the effectiveness under both transductive and inductive learning settings.