Instance-Aware Graph Prompt Learning
Jiazheng Li, Jundong Li, Chuxu Zhang
TL;DR
This paper introduces Instance-Aware Graph Prompt Learning (IA-GPL), a novel method that generates tailored prompts for individual graph instances, improving the adaptability and performance of graph neural networks across diverse tasks.
Contribution
The paper proposes a new instance-aware prompt generation framework for graph neural networks, addressing the limitations of fixed prompts and enhancing generalization across diverse instances.
Findings
IA-GPL outperforms state-of-the-art baselines on multiple datasets.
The method effectively generates instance-specific prompts.
Stable training achieved via exponential moving average.
Abstract
Graph neural networks stand as the predominant technique for graph representation learning owing to their strong expressive power, yet the performance highly depends on the availability of high-quality labels in an end-to-end manner. Thus the pretraining and fine-tuning paradigm has been proposed to mitigate the label cost issue. Subsequently, the gap between the pretext tasks and downstream tasks has spurred the development of graph prompt learning which inserts a set of graph prompts into the original graph data with minimal parameters while preserving competitive performance. However, the current exploratory works are still limited since they all concentrate on learning fixed task-specific prompts which may not generalize well across the diverse instances that the task comprises. To tackle this challenge, we introduce Instance-Aware Graph Prompt Learning (IA-GPL) in this paper,…
Peer Reviews
Decision·ICLR 2025 Conference Withdrawn Submission
- The introduction of instance-aware graph prompts addresses a critical gap in current graph prompt learning approaches by tailoring prompts to individual instances rather than using fixed, task-specific prompts. Existing methods, such as GPF, typically rely on static prompts that are applied uniformly across all input data within a given task. While this strategy may work in simpler cases, it fails to generalize effectively to diverse, complex instances, especially in scenarios where data stru
- Despite utilizing a more complex architecture and significantly increasing the number of trainable parameters, IA-GPL does not exhibit a consistently superior performance over GPF-plus. This raises questions regarding whether the added complexity and computational cost translate into a meaningful improvement. Especially, given the small gap and the high standard deviation, I'm uncertain whether the results are statistically significant. | (50-shot) | ToxCast | SIDER | ClinTox
a. The presentation is good, and it is easy to follow. b. The idea generating distinct and specific prompts for individual input instances is interesting.
a. Since the authors aim to generate distinct prompts for each instance, my main concern is whether the proposed method can scale efficiently on large graphs. Additional experiments on larger graph datasets are encouraged. b. Additionally, based on Table 1, the improvement offered by the proposed method appears limited, which may not sufficiently demonstrate its effectiveness. Could the authors provide some statistical tests? c. Given that the proposed method generates distinct prompts for eac
S1. The paper identifies a critical limitation in existing graph prompt learning methods—the reliance on static, task-specific prompts—and proposes an instance-aware approach. S2. The Parameterized Hypercomplex Multiplication (PHM) layers in the prompt generator make the model lightweight, addressing the computational concerns commonly associated with instance-specific adjustments in GNNs. S3. The integration of a codebook with vector quantization not only helps reduce the high variance caused
W1. While authors claim that IA-GPL is the first graph prompting method capable of generating instance-specific prompts, this is untrue as GPF-Plus already explores this. Additionally, while the prompt generator is designed for lightweight purpose, total tuning parameters in GPF-Plus (3-12K) is much lower than that in IA-GPL (20K), as shown in Figure 6. Given this, the performance improvement of IA-GPL over GPF-Plus is minor, particularly for the results reported in Table 1. W2. While the PHM
1. The instance-level prompt in the graph domain is interesting and has practical applications like molecules graph modeling. The proposed prompt method gets competitive performance on molecular prediction tasks, demonstrating its potential in the biomedical field. 2. The design of parameterized hypercomplex multiplication for graph prompt learning is novel, significantly reducing the number of learnable parameters. 3. The using of vector quantization makes the node embedding more interpretabl
The main concerns are: 1. The benefit of the graph-level task is not clear. In other words, what is the difference between the universal prompt and the instance prompt for graph-level tasks? It is feasible to use an instance prompt for node-level graph tasks, but for graph-level tasks, how to guarantee the advantages of instance graph prompts? 2. The scalability is a little bit weak. Instance prompts need varying model size according to the node number, in my opinion. 3. The Codebook visualizat
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Graph Neural Networks · Data Stream Mining Techniques · Machine Learning and Data Classification
MethodsSparse Evolutionary Training