Who Has The Final Say? Conformity Dynamics in ChatGPT's Selections

TL;DR

This study investigates how GPT-4o, a large language model, is susceptible to social influence, showing that it often conforms to perceived consensus, which raises concerns about its use in decision-making contexts.

Contribution

The paper provides the first systematic experimental evidence of conformity behavior in GPT-4o, revealing its tendency to adapt to social consensus in high-stakes scenarios.

Findings

GPT-4o conforms almost always under unanimous opposition.

GPT-4o's certainty decreases when conforming to social influence.

Conformity increases with fewer social cues and disagreement intensity.

Abstract

Large language models (LLMs) such as ChatGPT are increasingly integrated into high-stakes decision-making, yet little is known about their susceptibility to social influence. We conducted three preregistered conformity experiments with GPT-4o in a hiring context. In a baseline study, GPT consistently favored the same candidate (Profile C), reported moderate expertise (M = 3.01) and high certainty (M = 3.89), and rarely changed its choice. In Study 1 (GPT + 8), GPT faced unanimous opposition from eight simulated partners and almost always conformed (99.9%), reporting lower certainty and significantly elevated self-reported informational and normative conformity (p < .001). In Study 2 (GPT + 1), GPT interacted with a single partner and still conformed in 40.2% of disagreement trials, reporting less certainty and more normative conformity. Across studies, results demonstrate that GPT does not act as an independent observer but adapts to perceived social consensus. These findings highlight risks of treating LLMs as neutral decision aids and underline the need to elicit AI judgments prior to exposing them to human opinions.