Is ChatGPT a Good Personality Recognizer? A Preliminary Study

TL;DR

This study evaluates ChatGPT's ability to recognize personality traits from text, showing promising results with prompting strategies, while also highlighting fairness issues and benefits for related tasks.

Contribution

It provides a preliminary assessment of ChatGPT's text-based personality recognition capabilities using novel prompting strategies, including level-oriented prompts.

Findings

ChatGPT with zero-shot chain-of-thought prompts performs well in personality recognition.

Level-oriented prompting narrows the performance gap with state-of-the-art models.

Fairness issues observed towards gender and age attributes.

Abstract

In recent years, personality has been regarded as a valuable personal factor being incorporated into numerous tasks such as sentiment analysis and product recommendation. This has led to widespread attention to text-based personality recognition task, which aims to identify an individual's personality based on given text. Considering that ChatGPT has recently exhibited remarkable abilities on various natural language processing tasks, we provide a preliminary evaluation of ChatGPT on text-based personality recognition task for generating effective personality data. Concretely, we employ a variety of prompting strategies to explore ChatGPT's ability in recognizing personality from given text, especially the level-oriented prompting strategy we designed for guiding ChatGPT in analyzing given text at a specified level. The experimental results on two representative real-world datasets reveal that ChatGPT with zero-shot chain-of-thought prompting exhibits impressive personality recognition ability and is capable to provide natural language explanations through text-based logical reasoning. Furthermore, by employing the level-oriented prompting strategy to optimize zero-shot chain-of-thought prompting, the performance gap between ChatGPT and corresponding state-of-the-art model has been narrowed even more. However, we observe that ChatGPT shows unfairness towards certain sensitive demographic attributes such as gender and age. Additionally, we discover that eliciting the personality recognition ability of ChatGPT helps improve its performance on personality-related downstream tasks such as sentiment classification and stress prediction.