Evaluating the Simulation of Human Personality-Driven Susceptibility to Misinformation with LLMs

TL;DR

This study assesses whether large language models can simulate human personality-driven differences in susceptibility to misinformation, revealing both promising capabilities and notable limitations in capturing psychological variability.

Contribution

It demonstrates that LLMs conditioned on Big-Five personality traits can partially replicate human patterns of misinformation susceptibility, highlighting systematic biases and informing future behavioral modeling.

Findings

Certain personality-misinformation links are reliably reproduced.

Some trait associations diverge, indicating biases in LLMs.

The study provides insights into modeling cognitive diversity in AI.

Abstract

Large language models (LLMs) make it possible to generate synthetic behavioural data at scale, offering an ethical and low-cost alternative to human experiments. Whether such data can faithfully capture psychological differences driven by personality traits, however, remains an open question. We evaluate the capacity of LLM agents, conditioned on Big-Five profiles, to reproduce personality-based variation in susceptibility to misinformation, focusing on news discernment, the ability to judge true headlines as true and false headlines as false. Leveraging published datasets in which human participants with known personality profiles rated headline accuracy, we create matching LLM agents and compare their responses to the original human patterns. Certain trait-misinformation associations, notably those involving Agreeableness and Conscientiousness, are reliably replicated, whereas others diverge, revealing systematic biases in how LLMs internalize and express personality. The results underscore both the promise and the limits of personality-aligned LLMs for behavioral simulation, and offer new insight into modeling cognitive diversity in artificial agents.