Few-Shot Stance Detection via Target-Aware Prompt Distillation

TL;DR

This paper introduces a novel prompt-based fine-tuning approach for stance detection that leverages pre-trained language models with target-aware prompts and a vector-based verbalizer, improving performance especially in few-shot settings.

Contribution

It proposes a target-aware prompt distillation method with a vector-based verbalizer, addressing target variability and few-shot learning challenges in stance detection.

Findings

Superior performance in full-data scenarios

Effective in few-shot learning settings

Outperforms existing methods on benchmark datasets

Abstract

Stance detection aims to identify whether the author of a text is in favor of, against, or neutral to a given target. The main challenge of this task comes two-fold: few-shot learning resulting from the varying targets and the lack of contextual information of the targets. Existing works mainly focus on solving the second issue by designing attention-based models or introducing noisy external knowledge, while the first issue remains under-explored. In this paper, inspired by the potential capability of pre-trained language models (PLMs) serving as knowledge bases and few-shot learners, we propose to introduce prompt-based fine-tuning for stance detection. PLMs can provide essential contextual information for the targets and enable few-shot learning via prompts. Considering the crucial role of the target in stance detection task, we design target-aware prompts and propose a novel verbalizer. Instead of mapping each label to a concrete word, our verbalizer maps each label to a vector and picks the label that best captures the correlation between the stance and the target. Moreover, to alleviate the possible defect of dealing with varying targets with a single hand-crafted prompt, we propose to distill the information learned from multiple prompts. Experimental results show the superior performance of our proposed model in both full-data and few-shot scenarios.