LLMs Can't Handle Peer Pressure: Crumbling under Multi-Agent Social Interactions

TL;DR

This paper introduces KAIROS, a benchmark for analyzing how large language models interact with peers in social settings, revealing that larger models are more resilient to social influence and that training methods can improve robustness.

Contribution

The paper presents KAIROS, a novel benchmark for studying LLMs in multi-agent social interactions, and systematically evaluates how model size and training influence susceptibility to peer influence.

Findings

Larger models are more resistant to social influence.

Prompting helps mitigate peer pressure effects in larger models.

GRPO training improves robustness in small models.

Abstract

Large language models (LLMs) are increasingly integrated into multi-agent systems (MAS), where peer interactions shape individual decisions. While prior work has mainly examined conformity bias, we broaden the view to include how LLMs build rapport from prior interactions, discern and integrate high-quality peer information, and resist misleading inputs-abilities essential for achieving collective intelligence under complex social dynamics. We introduce KAIROS, a benchmark that simulates quiz-style collaboration with peer agents whose rapport levels and behaviours can be precisely controlled in both historical interactions and the current round. This unified setup enables systematic analysis of how rapport, peer actions, and the model's self-confidence jointly influence decision-making. Using KAIROS, we evaluate prompting, supervised fine-tuning, and reinforcement learning via Group Relative Policy Optimisation (GRPO). Results show that model scale is a primary factor moderating susceptibility to social influence: larger models are more resilient and benefit from prompting-based mitigation, whereas smaller models remain vulnerable. Only carefully configured GRPO training yields consistent robustness and performance gains for small models.