Language Models Trained on Media Diets Can Predict Public Opinion

TL;DR

This paper presents a novel method using media-trained language models to predict public opinion, validated against survey data, showing high accuracy especially for media-engaged populations and offering new tools for media effect analysis.

Contribution

Introduces media diet-adapted language models that can emulate public opinions based on media consumption, providing a new approach to studying media effects on society.

Findings

Predictive of survey responses across different media channels

More accurate for individuals with higher media engagement

Aligned with existing literature on media influence on opinions

Abstract

Public opinion reflects and shapes societal behavior, but the traditional survey-based tools to measure it are limited. We introduce a novel approach to probe media diet models -- language models adapted to online news, TV broadcast, or radio show content -- that can emulate the opinions of subpopulations that have consumed a set of media. To validate this method, we use as ground truth the opinions expressed in U.S. nationally representative surveys on COVID-19 and consumer confidence. Our studies indicate that this approach is (1) predictive of human judgements found in survey response distributions and robust to phrasing and channels of media exposure, (2) more accurate at modeling people who follow media more closely, and (3) aligned with literature on which types of opinions are affected by media consumption. Probing language models provides a powerful new method for investigating media effects, has practical applications in supplementing polls and forecasting public opinion, and suggests a need for further study of the surprising fidelity with which neural language models can predict human responses.