Enhancing Debunking Effectiveness through LLM-based Personality Adaptation

TL;DR

This paper introduces a method for creating personalized fake news debunking messages using LLMs guided by personality traits, improving persuasiveness and tailoring content to individual profiles.

Contribution

It presents a novel LLM prompting technique for personality-adapted debunking messages and an automated evaluation method to assess their effectiveness.

Findings

Personalized messages are more persuasive than generic ones.

Openness increases persuadability, Neuroticism decreases it.

Using multiple LLM evaluators yields more reliable assessments.

Abstract

This study proposes a novel methodology for generating personalized fake news debunking messages by prompting Large Language Models (LLMs) with persona-based inputs aligned to the Big Five personality traits: Extraversion, Agreeableness, Conscientiousness, Neuroticism, and Openness. Our approach guides LLMs to transform generic debunking content into personalized versions tailored to specific personality profiles. To assess the effectiveness of these transformations, we employ a separate LLM as an automated evaluator simulating corresponding personality traits, thereby eliminating the need for costly human evaluation panels. Our results show that personalized messages are generally seen as more persuasive than generic ones. We also find that traits like Openness tend to increase persuadability, while Neuroticism can lower it. Differences between LLM evaluators suggest that using multiple models provides a clearer picture. Overall, this work demonstrates a practical way to create more targeted debunking messages exploiting LLMs, while also raising important ethical questions about how such technology might be used.