Can Language Models Reason about Individualistic Human Values and Preferences?

TL;DR

This paper introduces IndieValueCatalog, a dataset for testing language models' ability to reason about individualistic human values, revealing current models' limitations and the importance of nuanced understanding beyond demographics.

Contribution

The paper presents IndieValueCatalog, a novel dataset for evaluating language models' capacity to understand individualistic values, and analyzes models' performance and biases in this context.

Findings

Frontier LMs achieve only 55-65% accuracy in predicting individualistic values.

Demographic information alone is insufficient to describe individualistic values.

Models show partiality in reasoning about global individualistic values, measured by the {}Inequity Index.

Abstract

Recent calls for pluralistic alignment emphasize that AI systems should address the diverse needs of all people. Yet, efforts in this space often require sorting people into fixed buckets of pre-specified diversity-defining dimensions (e.g., demographics), risking smoothing out individualistic variations or even stereotyping. To achieve an authentic representation of diversity that respects individuality, we propose individualistic alignment. While individualistic alignment can take various forms, we introduce IndieValueCatalog, a dataset transformed from the influential World Values Survey (WVS), to study language models (LMs) on the specific challenge of individualistic value reasoning. Given a sample of an individual's value-expressing statements, models are tasked with predicting this person's value judgments in novel cases. With IndieValueCatalog, we reveal critical limitations in frontier LMs, which achieve only 55 % to 65% accuracy in predicting individualistic values. Moreover, our results highlight that a precise description of individualistic values cannot be approximated only with demographic information. We also identify a partiality of LMs in reasoning about global individualistic values, as measured by our proposed Value Inequity Index ({\sigma}Inequity). Finally, we train a series of IndieValueReasoners to reveal new patterns and dynamics into global human values.