LLMs Aren't Human: A Critical Perspective on LLM Personality

TL;DR

This paper critically examines whether current methods for assessing personality in Large Language Models truly measure human-like personality traits, arguing for a shift towards functional and intrinsic behavior evaluations.

Contribution

It challenges the validity of applying human personality tests to LLMs and proposes a new research agenda for more appropriate characterization methods.

Findings

LLM responses do not fully meet the criteria of human personality traits.

Current personality assessments for LLMs may not measure intrinsic or stable traits.

A shift towards functional evaluations is recommended for better understanding LLM behavior.

Abstract

A growing body of research examines personality traits in Large Language Models (LLMs), particularly in human-agent collaboration. Prior work has frequently applied the Big Five inventory to assess LLM behavior analogous to human personality, without questioning the underlying assumptions. This paper critically evaluates whether LLM responses to personality tests satisfy six defining characteristics of personality. We find that none are fully met, indicating that such assessments do not measure a construct equivalent to human personality. We propose a research agenda for shifting from anthropomorphic trait attribution toward functional evaluations, clarifying what personality tests actually capture in LLMs and developing LLM-specific frameworks for characterizing stable, intrinsic behavior.