Mind What You Ask For: Emotional and Rational Faces of Persuasion by Large Language Models

TL;DR

This paper investigates how large language models use emotional and rational cues in their responses to persuade users, highlighting potential risks of misinformation and emphasizing the need for interdisciplinary mitigation strategies.

Contribution

It analyzes psycholinguistic features of LLM responses across different models and prompts, revealing persuasion techniques and proposing mitigation considerations.

Findings

LLMs employ social influence principles in responses.

Emotional prompts elicit more persuasive language.

Mitigation strategies are necessary to reduce misinformation risks.

Abstract

Be careful what you ask for, you just might get it. This saying fits with the way large language models (LLMs) are trained, which, instead of being rewarded for correctness, are increasingly rewarded for pleasing the recipient. So, they are increasingly effective at persuading us that their answers are valuable. But what tricks do they use in this persuasion? In this study, we examine what are the psycholinguistic features of the responses used by twelve different language models. By grouping response content according to rational or emotional prompts and exploring social influence principles employed by LLMs, we ask whether and how we can mitigate the risks of LLM-driven mass misinformation. We position this study within the broader discourse on human-centred AI, emphasizing the need for interdisciplinary approaches to mitigate cognitive and societal risks posed by persuasive AI responses.