Beyond One-Way Influence: Bidirectional Opinion Dynamics in Multi-Turn Human-LLM Interactions

TL;DR

This study explores the bidirectional influence in multi-turn human-LLM interactions, revealing how user input and personalization affect opinion shifts and LLM responses over conversations.

Contribution

It extends prior research by analyzing how user input impacts LLM responses and how personalization amplifies opinion dynamics in multi-turn dialogues.

Findings

Human opinions showed minimal change.

LLM outputs varied more significantly, reducing stance gaps.

Personalization increased the magnitude of opinion shifts.

Abstract

Large language model (LLM)-powered chatbots are increasingly used for opinion exploration. Prior research examined how LLMs alter user views, yet little work extended beyond one-way influence to address how user input can affect LLM responses and how such bi-directional influence manifests throughout the multi-turn conversations. This study investigates this dynamic through 50 controversial-topic discussions with participants (N=266) across three conditions: static statements, standard chatbot, and personalized chatbot. Results show that human opinions barely shifted, while LLM outputs changed more substantially, narrowing the gap between human and LLM stance. Personalization amplified these shifts in both directions compared to the standard setting. Analysis of multi-turn conversations further revealed that exchanges involving participants' personal stories were most likely to trigger stance changes for both humans and LLMs. Our work highlights the risk of over-alignment in human-LLM interaction and the need for careful design of personalized chatbots to more thoughtfully and stably align with users.