Reliable and Compact Graph Fine-tuning via GraphSparse Prompting

TL;DR

This paper introduces Graph Sparse Prompting (GSP), a method that adaptively selects key graph elements for efficient and compact fine-tuning of pre-trained GNNs, improving performance on multiple benchmarks.

Contribution

It proposes a novel sparse prompting framework for GNNs, enabling attribute selection and compact tuning, which is more efficient than existing all-element prompting methods.

Findings

Effective on 16 benchmark datasets

Achieves more compact prompts with maintained or improved accuracy

Demonstrates advantages over existing prompting methods

Abstract

Recently, graph prompt learning has garnered increasing attention in adapting pre-trained GNN models for downstream graph learning tasks. However, existing works generally conduct prompting over all graph elements (e.g., nodes, edges, node attributes, etc.), which is suboptimal and obviously redundant. To address this issue, we propose exploiting sparse representation theory for graph prompting and present Graph Sparse Prompting (GSP). GSP aims to adaptively and sparsely select the optimal elements (e.g., certain node attributes) to achieve compact prompting for downstream tasks. Specifically, we propose two kinds of GSP models, termed Graph Sparse Feature Prompting (GSFP) and Graph Sparse multi-Feature Prompting (GSmFP). Both GSFP and GSmFP provide a general scheme for tuning any specific pre-trained GNNs that can achieve attribute selection and compact prompt learning simultaneously. A simple yet effective algorithm has been designed for solving GSFP and GSmFP models. Experiments on 16 widely-used benchmark datasets validate the effectiveness and advantages of the proposed GSFPs.