Is Explanation the Cure? Misinformation Mitigation in the Short Term and Long Term

TL;DR

This study compares warning labels and GPT-4 generated explanations in reducing belief in misinformation, finding both methods equally effective in short-term and long-term scenarios through human-subject experiments.

Contribution

It provides empirical evidence on the effectiveness of explanations versus warning labels in misinformation mitigation, highlighting the potential of NLP-generated explanations.

Findings

Both interventions significantly reduced belief in fake news.

Warning labels and explanations are equally effective long-term.

NLP explanations can complement traditional misinformation strategies.

Abstract

With advancements in natural language processing (NLP) models, automatic explanation generation has been proposed to mitigate misinformation on social media platforms in addition to adding warning labels to identified fake news. While many researchers have focused on generating good explanations, how these explanations can really help humans combat fake news is under-explored. In this study, we compare the effectiveness of a warning label and the state-of-the-art counterfactual explanations generated by GPT-4 in debunking misinformation. In a two-wave, online human-subject study, participants (N = 215) were randomly assigned to a control group in which false contents are shown without any intervention, a warning tag group in which the false claims were labeled, or an explanation group in which the false contents were accompanied by GPT-4 generated explanations. Our results show that both interventions significantly decrease participants' self-reported belief in fake claims in an equivalent manner for the short-term and long-term. We discuss the implications of our findings and directions for future NLP-based misinformation debunking strategies.