Can AI Understand Human Personality? -- Comparing Human Experts and AI Systems at Predicting Personality Correlations

TL;DR

This study compares AI models and humans in predicting human personality correlations, finding that AI can outperform laypeople but specialized models still rival experts, highlighting strengths and limitations of current AI systems.

Contribution

The paper demonstrates that specialized neural networks outperform most humans in predicting personality correlations, while large language models excel in certain aspects, emphasizing the complementary roles of different AI approaches.

Findings

AI models outperform most laypeople and experts in predicting correlations.

Specialized models like PersonalityMap match or exceed expert performance.

Large language models outperform most individuals but lag behind experts on some measures.

Abstract

We test the abilities of specialised deep neural networks like PersonalityMap as well as general LLMs like GPT-4o and Claude 3 Opus in understanding human personality. Specifically, we compare their ability to predict correlations between personality items to the abilities of lay people and academic experts. We find that when compared with individual humans, all AI models make better predictions than the vast majority of lay people and academic experts. However, when selecting the median prediction for each item, we find a different pattern: Experts and PersonalityMap outperform LLMs and lay people on most measures. Our results suggest that while frontier LLMs' are better than most individual humans at predicting correlations between personality items, specialised models like PersonalityMap continue to match or exceed expert human performance even on some outcome measures where LLMs underperform. This provides evidence both in favour of the general capabilities of large language models and in favour of the continued place for specialised models trained and deployed for specific domains.