Collaborative Stance Detection via Small-Large Language Model Consistency Verification

TL;DR

This paper introduces CoVer, a framework that combines small and large language models to improve stance detection on social media by verifying consistency, reducing reliance on costly LLMs, and enhancing performance in zero-shot settings.

Contribution

The paper proposes a novel collaborative framework that integrates SLMs and LLMs for stance detection, emphasizing context sharing and logical verification to improve accuracy and efficiency.

Findings

Outperforms state-of-the-art methods on multiple benchmarks.

Achieves 0.54 LLM queries per tweet, reducing costs.

Significantly improves zero-shot stance detection performance.

Abstract

Stance detection on social media aims to identify attitudes expressed in tweets towards specific targets. Current studies prioritize Large Language Models (LLMs) over Small Language Models (SLMs) due to the overwhelming performance improving provided by LLMs. However, heavily relying on LLMs for stance detection, regardless of the cost, is impractical for real-world social media monitoring systems that require vast data analysis. To this end, we propose \textbf{\underline{Co}}llaborative Stance Detection via Small-Large Language Model Consistency \textbf{\underline{Ver}}ification (\textbf{CoVer}) framework, which enhances LLM utilization via context-shared batch reasoning and logical verification between LLM and SLM. Specifically, instead of processing each text individually, CoVer processes texts batch-by-batch, obtaining stance predictions and corresponding explanations via LLM reasoning in a shared context. Then, to exclude the bias caused by context noises, CoVer introduces the SLM for logical consistency verification. Finally, texts that repeatedly exhibit low logical consistency are classified using consistency-weighted aggregation of prior LLM stance predictions. Our experiments show that CoVer outperforms state-of-the-art methods across multiple benchmarks in the zero-shot setting, achieving 0.54 LLM queries per tweet while significantly enhancing performance. Our CoVer offers a more practical solution for LLM deploying for social media stance detection.