GFMate: Empowering Graph Foundation Models with Test-time Prompt Tuning

TL;DR

GFMate introduces a novel test-time prompt tuning method for Graph Foundation Models that enhances domain generalization and leverages unlabelled data, achieving significant performance improvements.

Contribution

It proposes a pre-training-agnostic prompt tuning approach with centroid and layer prompts, and a test-time learning objective utilizing unlabelled data.

Findings

Achieves up to 30.63% performance improvement on 12 datasets.

Demonstrates superior efficiency and generalizability of GFMate.

Effectively exploits unlabelled target data for prompt tuning.

Abstract

Graph prompt tuning has shown great potential in graph learning by introducing trainable prompts to enhance the model performance in conventional single-domain scenarios. Recent research has extended graph prompts to improve Graph Foundation Models (GFMs) by few-shot tuning auxiliary prompts. Despite their progress, most existing methods embed source-domain information into prompts, which serve either as input to GFMs or encoded during model pre-training. Such prompt entanglement with specific source domains and GFM pre-training strategy restricts their generalisability to other domains and different GFMs. Furthermore, existing GFM prompts merely rely on few-shot tuning for adaptation, neglecting the rich information in unlabelled target domain test data. Motivated by these insights, this paper aims to empower GFMs with pre-training-agnostic test-time graph prompt tuning, named GFMate. GFMate introduces centroid and layer prompts applied after pre-training on target domains, avoiding entanglement with specific source domains and model pre-training. In addition, a test-time complementary learning objective is devised to exploit both labelled and unlabelled target domain data for effective test-time prompt tuning. Extensive experiments on 12 benchmark datasets demonstrate the superior performance and efficiency of GFMate, achieving improvements of up to 30.63%. Code is available at https://github.com/YanJiangJerry/GFMate.