Using LLMs to Model the Beliefs and Preferences of Targeted Populations

TL;DR

This paper explores how to fine-tune large language models to accurately simulate the beliefs and preferences of specific human populations, enabling applications like virtual surveys and behavioral testing.

Contribution

It benchmarks two fine-tuning methods for LLMs to model population preferences and introduces a novel loss term to enhance numeric response accuracy.

Findings

Fine-tuning improves population preference modeling.

Temperature affects the trade-off between individual and population-level accuracy.

The new loss term enhances numeric response modeling.

Abstract

We consider the problem of aligning a large language model (LLM) to model the preferences of a human population. Modeling the beliefs, preferences, and behaviors of a specific population can be useful for a variety of different applications, such as conducting simulated focus groups for new products, conducting virtual surveys, and testing behavioral interventions, especially for interventions that are expensive, impractical, or unethical. Existing work has had mixed success using LLMs to accurately model human behavior in different contexts. We benchmark and evaluate two well-known fine-tuning approaches and evaluate the resulting populations on their ability to match the preferences of real human respondents on a survey of preferences for battery electric vehicles (BEVs). We evaluate our models against their ability to match population-wide statistics as well as their ability to match individual responses, and we investigate the role of temperature in controlling the trade-offs between these two. Additionally, we propose and evaluate a novel loss term to improve model performance on responses that require a numeric response.