From Biased Chatbots to Biased Agents: Examining Role Assignment Effects on LLM Agent Robustness

TL;DR

This paper demonstrates that demographic-based persona assignments can significantly impair LLM agent performance and introduce biases, highlighting a critical vulnerability in deploying reliable autonomous AI agents.

Contribution

It provides the first systematic analysis of how persona-induced biases affect LLM agent robustness across multiple domains and models.

Findings

Performance degradation up to 26.2% due to persona cues

Biases influence decision-making across diverse tasks

Persona conditioning can distort agent reliability

Abstract

Large Language Models (LLMs) are increasingly deployed as autonomous agents capable of actions with real-world impacts beyond text generation. While persona-induced biases in text generation are well documented, their effects on agent task performance remain largely unexplored, even though such effects pose more direct operational risks. In this work, we present the first systematic case study showing that demographic-based persona assignments can alter LLM agents' behavior and degrade performance across diverse domains. Evaluating widely deployed models on agentic benchmarks spanning strategic reasoning, planning, and technical operations, we uncover substantial performance variations - up to 26.2% degradation, driven by task-irrelevant persona cues. These shifts appear across task types and model architectures, indicating that persona conditioning and simple prompt injections can distort an agent's decision-making reliability. Our findings reveal an overlooked vulnerability in current LLM agentic systems: persona assignments can introduce implicit biases and increase behavioral volatility, raising concerns for the safe and robust deployment of LLM agents.