Tutorials on Stance Detection using Pre-trained Language Models: Fine-tuning BERT and Prompting Large Language Models

TL;DR

This paper offers comprehensive tutorials on stance detection in Twitter data, comparing fine-tuning BERT with prompting large language models like ChatGPT and FLAN-T5, highlighting their respective strengths.

Contribution

It provides practical, step-by-step tutorials on both fine-tuning BERT and prompting LLMs for stance detection, including implementation details and performance evaluation.

Findings

Few-shot prompting of ChatGPT and FLAN-T5 outperforms fine-tuned BERT models.

Tutorials include code, visualizations, and insights for applied stance detection.

Strategies for constructing effective prompts are evaluated and demonstrated.

Abstract

This paper presents two self-contained tutorials on stance detection in Twitter data using BERT fine-tuning and prompting large language models (LLMs). The first tutorial explains BERT architecture and tokenization, guiding users through training, tuning, and evaluating standard and domain-specific BERT models with HuggingFace transformers. The second focuses on constructing prompts and few-shot examples to elicit stances from ChatGPT and open-source FLAN-T5 without fine-tuning. Various prompting strategies are implemented and evaluated using confusion matrices and macro F1 scores. The tutorials provide code, visualizations, and insights revealing the strengths of few-shot ChatGPT and FLAN-T5 which outperform fine-tuned BERTs. By covering both model fine-tuning and prompting-based techniques in an accessible, hands-on manner, these tutorials enable learners to gain applied experience with cutting-edge methods for stance detection.