Large Language Models (LLMs) as Agents for Augmented Democracy

TL;DR

This paper demonstrates that fine-tuned large language models can accurately predict individual and aggregate political preferences, enhancing data quality for augmented democratic decision-making.

Contribution

It introduces a method of using fine-tuned LLMs to augment citizen preference data, improving prediction accuracy over traditional models in a real-world electoral context.

Findings

LLMs outperform simple bundle rule in individual preference prediction

Augmented samples yield more accurate population preference estimates

LLMs capture nuanced policy preferences beyond party lines

Abstract

We explore an augmented democracy system built on off-the-shelf LLMs fine-tuned to augment data on citizen's preferences elicited over policies extracted from the government programs of the two main candidates of Brazil's 2022 presidential election. We use a train-test cross-validation setup to estimate the accuracy with which the LLMs predict both: a subject's individual political choices and the aggregate preferences of the full sample of participants. At the individual level, we find that LLMs predict out of sample preferences more accurately than a "bundle rule", which would assume that citizens always vote for the proposals of the candidate aligned with their self-reported political orientation. At the population level, we show that a probabilistic sample augmented by an LLM provides a more accurate estimate of the aggregate preferences of a population than the non-augmented probabilistic sample alone. Together, these results indicates that policy preference data augmented using LLMs can capture nuances that transcend party lines and represents a promising avenue of research for data augmentation.