Can LLM Agents Maintain a Persona in Discourse?

TL;DR

This paper investigates whether large language models can sustain consistent, personality-aligned discourse by analyzing their ability to maintain psychological traits during conversations, revealing significant variability and challenges.

Contribution

It introduces a framework for testing LLMs' ability to maintain and infer personality traits in dialogue, highlighting the variability and limitations in current models.

Findings

LLMs can be guided to exhibit personality traits in dialogue

Model and setting combinations significantly affect personality consistency

Inconsistencies challenge the development of stable personality-aligned LLMs

Abstract

Large Language Models (LLMs) are widely used as conversational agents, exploiting their capabilities in various sectors such as education, law, medicine, and more. However, LLMs are often subjected to context-shifting behaviour, resulting in a lack of consistent and interpretable personality-aligned interactions. Adherence to psychological traits lacks comprehensive analysis, especially in the case of dyadic (pairwise) conversations. We examine this challenge from two viewpoints, initially using two conversation agents to generate a discourse on a certain topic with an assigned personality from the OCEAN framework (Openness, Conscientiousness, Extraversion, Agreeableness, and Neuroticism) as High/Low for each trait. This is followed by using multiple judge agents to infer the original traits assigned to explore prediction consistency, inter-model agreement, and alignment with the assigned personality. Our findings indicate that while LLMs can be guided toward personality-driven dialogue, their ability to maintain personality traits varies significantly depending on the combination of models and discourse settings. These inconsistencies emphasise the challenges in achieving stable and interpretable personality-aligned interactions in LLMs.