Prompt-Based Zero- and Few-Shot Node Classification: A Multimodal Approach

TL;DR

This paper introduces TAG, a multimodal approach combining raw text and graph data for zero- and few-shot node classification, outperforming existing methods without meta-learning.

Contribution

The paper proposes a novel two-stage multimodal model, TAG, that effectively integrates raw text and graph topology for limited-label node classification tasks.

Findings

TAG outperforms baselines in zero-shot classification

TAG achieves significant improvements in few-shot scenarios

The approach effectively leverages raw text and graph data without meta-learning

Abstract

Multimodal data empowers machine learning models to better understand the world from various perspectives. In this work, we study the combination of \emph{text and graph} modalities, a challenging but understudied combination which is prevalent across multiple settings including citation networks, social media, and the web. We focus on the popular task of node classification using limited labels; in particular, under the zero- and few-shot scenarios. In contrast to the standard pipeline which feeds standard precomputed (e.g., bag-of-words) text features into a graph neural network, we propose \textbf{T}ext-\textbf{A}nd-\textbf{G}raph (TAG) learning, a more deeply multimodal approach that integrates the raw texts and graph topology into the model design, and can effectively learn from limited supervised signals without any meta-learning procedure. TAG is a two-stage model with (1) a prompt- and graph-based module which generates prior logits that can be directly used for zero-shot node classification, and (2) a trainable module that further calibrates these prior logits in a few-shot manner. Experiments on two node classification datasets show that TAG outperforms all the baselines by a large margin in both zero- and few-shot settings.