Is Self-knowledge and Action Consistent or Not: Investigating Large Language Model's Personality

TL;DR

This paper investigates whether large language models' self-reported personality traits align with their actual behavior, highlighting potential discrepancies between self-knowledge and actions in AI systems.

Contribution

It introduces a framework for evaluating the consistency between LLMs' self-assessed personalities and their real-world responses, revealing insights into their self-awareness.

Findings

LLMs often show discrepancies between self-reported traits and observed behaviors.

Personality questionnaires may not fully capture LLMs' complex response patterns.

The study highlights the need for better evaluation methods for AI personality assessment.

Abstract

In this study, we delve into the validity of conventional personality questionnaires in capturing the human-like personality traits of Large Language Models (LLMs). Our objective is to assess the congruence between the personality traits LLMs claim to possess and their demonstrated tendencies in real-world scenarios. By conducting an extensive examination of LLM outputs against observed human response patterns, we aim to understand the disjunction between self-knowledge and action in LLMs.