ValueSim: Generating Backstories to Model Individual Value Systems

TL;DR

ValueSim is a novel framework that generates personalized backstories to simulate individual human value systems, improving alignment of language models with personal values through narrative-based modeling.

Contribution

It introduces a multi-module architecture inspired by the Cognitive-Affective Personality System to generate and utilize personal narratives for value simulation.

Findings

Over 10% improvement in top-1 accuracy over retrieval-augmented methods

Performance increases with more user interaction history

Demonstrates effective simulation of individual value systems

Abstract

As Large Language Models (LLMs) continue to exhibit increasingly human-like capabilities, aligning them with human values has become critically important. Contemporary advanced techniques, such as prompt learning and reinforcement learning, are being deployed to better align LLMs with human values. However, while these approaches address broad ethical considerations and helpfulness, they rarely focus on simulating individualized human value systems. To address this gap, we present ValueSim, a framework that simulates individual values through the generation of personal backstories reflecting past experiences and demographic information. ValueSim converts structured individual data into narrative backstories and employs a multi-module architecture inspired by the Cognitive-Affective Personality System to simulate individual values based on these narratives. Testing ValueSim on a self-constructed benchmark derived from the World Values Survey demonstrates an improvement in top-1 accuracy by over 10% compared to retrieval-augmented generation methods. Further analysis reveals that performance enhances as additional user interaction history becomes available, indicating the model's ability to refine its persona simulation capabilities over time.