Zero-Shot Stance Detection using Contextual Data Generation with LLMs

TL;DR

This paper introduces DyMoAdapt, a method that uses GPT-3 to generate topic-specific data for improving zero-shot stance detection, and presents the MGT-VAST dataset for multi-topic context understanding.

Contribution

It proposes a novel approach combining data generation with LLMs for zero-shot stance detection and introduces a new multi-topic dataset to facilitate research.

Findings

Generated data did not significantly improve performance.

The MGT-VAST dataset enables multi-topic context analysis.

The approach highlights challenges in data augmentation for stance detection.

Abstract

Stance detection, the classification of attitudes expressed in a text towards a specific topic, is vital for applications like fake news detection and opinion mining. However, the scarcity of labeled data remains a challenge for this task. To address this problem, we propose Dynamic Model Adaptation with Contextual Data Generation (DyMoAdapt) that combines Few-Shot Learning and Large Language Models. In this approach, we aim to fine-tune an existing model at test time. We achieve this by generating new topic-specific data using GPT-3. This method could enhance performance by allowing the adaptation of the model to new topics. However, the results did not increase as we expected. Furthermore, we introduce the Multi Generated Topic VAST (MGT-VAST) dataset, which extends VAST using GPT-3. In this dataset, each context is associated with multiple topics, allowing the model to understand the relationship between contexts and various potential topics