Designing Domain-Specific Large Language Models: The Critical Role of Fine-Tuning in Public Opinion Simulation

TL;DR

This paper presents a fine-tuning method for large language models that incorporates socio-demographic data to improve the simulation of public opinions on environmental policies, demonstrating enhanced accuracy and demographic representation.

Contribution

It introduces a novel fine-tuning approach using profiling factors from socio-demographic data to better simulate diverse opinions in LLMs, outperforming pre-trained models.

Findings

Enhanced demographic accuracy in opinion simulation

Significant improvements in evaluation metrics

Potential applications in policy and societal domains

Abstract

Large language models (LLMs) have transformed natural language processing, yet face challenges in specialized tasks such as simulating opinions on environmental policies. This paper introduces a novel fine-tuning approach that integrates socio-demographic data from the UK Household Longitudinal Study, uniquely using profiling factors, such as age, gender, income, education, and region. This method enhances the accuracy and representation of generated views. By emulating diverse synthetic profiles, the fine-tuned models significantly outperform pre-trained counterparts, achieving measurable improvements in capturing demographic nuances. Evaluation metrics, including Chi-Squared, Cosine Similarity, Jaccard Index, and KL-divergence, reveal a strong alignment between synthetic and real-world opinions. This work demonstrates the potential of fine-tuned LLMs tailored to societal contexts to enable more ethical and precise policy simulations. Its broader implications include deploying LLMs in domains like healthcare and education, fostering inclusive and data-driven decision-making in both research and practice.