Aligning Language Models to User Opinions

TL;DR

This paper explores how to better align large language models with individual user opinions by modeling personal opinions, demographics, and ideologies, resulting in improved prediction accuracy of user preferences.

Contribution

It introduces a novel approach that combines user opinions with demographics and ideology to enhance LLM alignment, surpassing traditional demographic-based prompting methods.

Findings

Up to 7 points accuracy improvement in predicting public opinions.

Mining past opinions improves prediction accuracy.

User opinions are not strongly predicted by demographics or ideology.

Abstract

An important aspect of developing LLMs that interact with humans is to align models' behavior to their users. It is possible to prompt an LLM into behaving as a certain persona, especially a user group or ideological persona the model captured during its pertaining stage. But, how to best align an LLM with a specific user and not a demographic or ideological group remains an open question. Mining public opinion surveys (by Pew Research), we find that the opinions of a user and their demographics and ideologies are not mutual predictors. We use this insight to align LLMs by modeling both user opinions as well as user demographics and ideology, achieving up to 7 points accuracy gains in predicting public opinions from survey questions across a broad set of topics. In addition to the typical approach of prompting LLMs with demographics and ideology, we discover that utilizing the most relevant past opinions from individual users enables the model to predict user opinions more accurately.