Evaluating and Inducing Personality in Pre-trained Language Models

TL;DR

This paper introduces a novel approach to evaluate and induce specific human-like personalities in large language models using psychometric tools based on the Big Five theory, enabling more nuanced understanding and control of machine behaviors.

Contribution

It presents the Machine Personality Inventory (MPI) for systematic personality assessment of LLMs and a Personality Prompting (P^2) method to controllably induce desired personalities.

Findings

MPI effectively evaluates LLM behaviors based on human personality tests.

P^2 can reliably induce specific personalities in LLMs.

The approach enables diverse and verifiable personality expressions in models.

Abstract

Standardized and quantified evaluation of machine behaviors is a crux of understanding LLMs. In this study, we draw inspiration from psychometric studies by leveraging human personality theory as a tool for studying machine behaviors. Originating as a philosophical quest for human behaviors, the study of personality delves into how individuals differ in thinking, feeling, and behaving. Toward building and understanding human-like social machines, we are motivated to ask: Can we assess machine behaviors by leveraging human psychometric tests in a principled and quantitative manner? If so, can we induce a specific personality in LLMs? To answer these questions, we introduce the Machine Personality Inventory (MPI) tool for studying machine behaviors; MPI follows standardized personality tests, built upon the Big Five Personality Factors (Big Five) theory and personality assessment inventories. By systematically evaluating LLMs with MPI, we provide the first piece of evidence demonstrating the efficacy of MPI in studying LLMs behaviors. We further devise a Personality Prompting (P^2) method to induce LLMs with specific personalities in a controllable way, capable of producing diverse and verifiable behaviors. We hope this work sheds light on future studies by adopting personality as the essential indicator for various downstream tasks, and could further motivate research into equally intriguing human-like machine behaviors.