AI Text-to-Behavior: A Study In Steerability

TL;DR

This study investigates how Large Language Models like ChatGPT can be guided to adopt specific personality traits using a behavioral psychology framework, revealing their ability to respond to nuanced prompts and simulate personas.

Contribution

Introduces a quantitative framework using OCEAN traits to measure and analyze LLM steerability and persona simulation capabilities.

Findings

GPT can distinguish and evoke specific personality traits.

Linguistic ambiguity varies across different OCEAN traits.

LLMs can internalize and replicate historical personas accurately.

Abstract

The research explores the steerability of Large Language Models (LLMs), particularly OpenAI's ChatGPT iterations. By employing a behavioral psychology framework called OCEAN (Openness, Conscientiousness, Extroversion, Agreeableness, Neuroticism), we quantitatively gauged the model's responsiveness to tailored prompts. When asked to generate text mimicking an extroverted personality, OCEAN scored the language alignment to that behavioral trait. In our analysis, while "openness" presented linguistic ambiguity, "conscientiousness" and "neuroticism" were distinctly evoked in the OCEAN framework, with "extroversion" and "agreeableness" showcasing a notable overlap yet distinct separation from other traits. Our findings underscore GPT's versatility and ability to discern and adapt to nuanced instructions. Furthermore, historical figure simulations highlighted the LLM's capacity to internalize and project instructible personas, precisely replicating their philosophies and dialogic styles. However, the rapid advancements in LLM capabilities and the opaque nature of some training techniques make metric proposals degrade rapidly. Our research emphasizes a quantitative role to describe steerability in LLMs, presenting both its promise and areas for further refinement in aligning its progress to human intentions.