Generative Psycho-Lexical Approach for Constructing Value Systems in Large Language Models

TL;DR

This paper introduces a new method called GPLA to construct psychologically grounded value systems for LLMs, improving their safety and alignment by integrating psychological principles into AI models.

Contribution

The paper presents the Generative Psycho-Lexical Approach (GPLA), a novel scalable method for creating value systems in LLMs based on psychological theories.

Findings

GPLA constructs a five-factor value system aligned with psychological criteria.

The proposed value system better captures LLM values than Schwartz's original model.

Using the new value system improves LLM safety prediction and alignment.

Abstract

Values are core drivers of individual and collective perception, cognition, and behavior. Value systems, such as Schwartz's Theory of Basic Human Values, delineate the hierarchy and interplay among these values, enabling cross-disciplinary investigations into decision-making and societal dynamics. Recently, the rise of Large Language Models (LLMs) has raised concerns regarding their elusive intrinsic values. Despite growing efforts in evaluating, understanding, and aligning LLM values, a psychologically grounded LLM value system remains underexplored. This study addresses the gap by introducing the Generative Psycho-Lexical Approach (GPLA), a scalable, adaptable, and theoretically informed method for constructing value systems. Leveraging GPLA, we propose a psychologically grounded five-factor value system tailored for LLMs. For systematic validation, we present three benchmarking tasks that integrate psychological principles with cutting-edge AI priorities. Our results reveal that the proposed value system meets standard psychological criteria, better captures LLM values, improves LLM safety prediction, and enhances LLM alignment, when compared to the canonical Schwartz's values.