Ladder-of-Thought: Using Knowledge as Steps to Elevate Stance Detection

TL;DR

The paper introduces Ladder-of-Thought, a method that enhances stance detection by guiding small language models to incorporate external knowledge, resulting in significant performance improvements over existing reasoning techniques like Chain-of-Thought.

Contribution

It proposes a dual-phase Progressive Optimization Framework that helps small LMs integrate external knowledge, improving reasoning and prediction accuracy in stance detection.

Findings

LoT improves stance detection accuracy by 16% over GPT-3.5.

LoT outperforms GPT-3.5 with CoT by 10%.

External knowledge integration enhances model reasoning.

Abstract

Stance detection aims to identify the attitude expressed in a document towards a given target. Techniques such as Chain-of-Thought (CoT) prompting have advanced this task, enhancing a model's reasoning capabilities through the derivation of intermediate rationales. However, CoT relies primarily on a model's pre-trained internal knowledge during reasoning, thereby neglecting the valuable external information that is previously unknown to the model. This omission, especially within the unsupervised reasoning process, can affect the model's overall performance. Moreover, while CoT enhances Large Language Models (LLMs), smaller LMs, though efficient operationally, face challenges in delivering nuanced reasoning. In response to these identified gaps, we introduce the Ladder-of-Thought (LoT) for the stance detection task. Constructed through a dual-phase Progressive Optimization Framework, LoT directs the small LMs to assimilate high-quality external knowledge, refining the intermediate rationales produced. These bolstered rationales subsequently serve as the foundation for more precise predictions - akin to how a ladder facilitates reaching elevated goals. LoT achieves a balance between efficiency and performance. Our empirical evaluations underscore LoT's efficacy, marking a 16% improvement over GPT-3.5 and a 10% enhancement compared to GPT-3.5 with CoT on stance detection task.