The Thin Line Between Comprehension and Persuasion in LLMs

TL;DR

This paper investigates whether large language models genuinely understand discourse by analyzing their persuasive abilities and comprehension of argumentative structures in debates.

Contribution

It provides empirical evidence of LLMs' persuasive skills and highlights their limitations in understanding deeper dialogical structures.

Findings

LLMs can maintain coherent, persuasive debates.

People become more critical when aware of AI involvement.

LLMs lack understanding of argument quality and supporting premises.

Abstract

Large language models (LLMs) are excellent at maintaining high-level, convincing dialogue, but it remains unclear whether their persuasive success reflects genuine understanding of the discourse. We examine this question through informal debates between humans and LLMs, first by measuring their persuasive skills, and then by relating these to their understanding of _what_ is being talked about: namely, their comprehension of argumentative structures and the pragmatic context on the same debates. We find that LLMs effectively maintain coherent, persuasive debates, and can sway the beliefs of both participants and audiences. We also note that awareness or suspicion of AI involvement encourage people to be more critical of the arguments made. However, we also find that LLMs are unable to show comprehension of deeper dialogical structures, such as argument quality or existence of supporting premises. Our results reveal a disconnect between LLM comprehension and dialogical skills, raising ethical and practical concerns on their deployment on explanation-critical contexts. From an argumentation-theoretical perspective, we experimentally question whether an agent, if it can convincingly maintain a dialogue, is required to show it knows what is talking about.