Pre-Training and Prompting for Few-Shot Node Classification on Text-Attributed Graphs

TL;DR

This paper introduces P2TAG, a novel framework that combines graph pre-training and prompting to improve few-shot node classification on text-attributed graphs, leveraging raw text and graph data.

Contribution

The paper proposes P2TAG, integrating self-supervised pre-training of language models and GNNs with a mixed prompt method for enhanced few-shot classification on TAGs.

Findings

Outperforms existing methods by 18.98% to 35.98% on real-world datasets.

Effectively utilizes raw text and graph information through pre-training and prompting.

Demonstrates significant improvements across six diverse TAG datasets.

Abstract

The text-attributed graph (TAG) is one kind of important real-world graph-structured data with each node associated with raw texts. For TAGs, traditional few-shot node classification methods directly conduct training on the pre-processed node features and do not consider the raw texts. The performance is highly dependent on the choice of the feature pre-processing method. In this paper, we propose P2TAG, a framework designed for few-shot node classification on TAGs with graph pre-training and prompting. P2TAG first pre-trains the language model (LM) and graph neural network (GNN) on TAGs with self-supervised loss. To fully utilize the ability of language models, we adapt the masked language modeling objective for our framework. The pre-trained model is then used for the few-shot node classification with a mixed prompt method, which simultaneously considers both text and graph information. We conduct experiments on six real-world TAGs, including paper citation networks and product co-purchasing networks. Experimental results demonstrate that our proposed framework outperforms existing graph few-shot learning methods on these datasets with +18.98% ~ +35.98% improvements.