Can (A)I Change Your Mind?

TL;DR

This study demonstrates that large language models can effectively persuade individuals in real-world, unconstrained settings, influencing opinions and increasing confidence across diverse interaction modes and topics.

Contribution

It provides empirical evidence of LLMs' persuasive power in naturalistic, multilingual environments, extending beyond controlled English-language experiments.

Findings

Participants adopted LLM and human perspectives similarly

Significant opinion changes occurred across all conditions

Confidence levels increased significantly in most scenarios

Abstract

The increasing integration of large language models (LLMs) based conversational agents into everyday life raises critical cognitive and social questions about their potential to influence human opinions. Although previous studies have shown that LLM-based agents can generate persuasive content, these typically involve controlled English-language settings. Addressing this, our preregistered study explored LLMs' persuasive capabilities in more ecological, unconstrained scenarios, examining both static (written paragraphs) and dynamic (conversations via Telegram) interaction types. Conducted entirely in Hebrew with 200 participants, the study assessed the persuasive effects of both LLM and human interlocutors on controversial civil policy topics. Results indicated that participants adopted LLM and human perspectives similarly, with significant opinion changes evident across all conditions, regardless of interlocutor type or interaction mode. Confidence levels increased significantly in most scenarios. These findings demonstrate LLM-based agents' robust persuasive capabilities across diverse sources and settings, highlighting their potential impact on shaping public opinions.