Projective Psychological Assessment of Large Multimodal Models Using Thematic Apperception Tests

TL;DR

This paper explores assessing personality traits of large multimodal models using a projective psychological framework, demonstrating that models can understand complex social and emotional concepts but struggle with aggression regulation.

Contribution

It introduces a novel method for evaluating LMMs' personality traits through TAT responses and assesses their social cognition using the SCORS-G framework.

Findings

Models understand interpersonal dynamics well

Models have a good grasp of self-concept

Models struggle with perceiving and regulating aggression

Abstract

Thematic Apperception Test (TAT) is a psychometrically grounded, multidimensional assessment framework that systematically differentiates between cognitive-representational and affective-relational components of personality-like functioning. This test is a projective psychological framework designed to uncover unconscious aspects of personality. This study examines whether the personality traits of Large Multimodal Models (LMMs) can be assessed through non-language-based modalities, using the Social Cognition and Object Relations Scale - Global (SCORS-G). LMMs are employed in two distinct roles: as subject models (SMs), which generate stories in response to TAT images, and as evaluator models (EMs), who assess these narratives using the SCORS-G framework. Evaluators demonstrated an excellent ability to understand and analyze TAT responses. Their interpretations are highly consistent with those of human experts. Assessment results highlight that all models understand interpersonal dynamics very well and have a good grasp of the concept of self. However, they consistently fail to perceive and regulate aggression. Performance varied systematically across model families, with larger and more recent models consistently outperforming smaller and earlier ones across SCORS-G dimensions.