Testing the validity of multiple opinion dynamics models

TL;DR

This study evaluates various opinion dynamics models against real-world data, revealing their limitations in predicting future opinions and highlighting the need for more accurate models for social issues.

Contribution

It introduces a standardized validation benchmark for opinion dynamics models using empirical data and demonstrates their current inadequacy in real-world predictions.

Findings

Models perform well on simulated data but fail on empirical data.

Most models tend to 'freeze' and replicate previous year's data.

Current models are incompatible with real-world opinion dynamics.

Abstract

While opinion dynamics models have been extensively studied as stylized models, there has been growing attention to the possibility of combining these models with empirical data. This attention seems to be driven by the many social issues that strongly depend on people's opinions (such as climate change and vaccination) and the need for empirically valid models to design related policy interventions. While different models have been combined in various ways with empirical data, standardised comparison of models against empirical data is still lacking. In this article, we test the validity of multiple opinion dynamics models--including both stylized and more realistic models. Our approach follows a "data science-like" validation procedure, where we first calibrate the model's free parameters using an initial range of years (e.g. 2010-2015), and then use data from one wave (e.g. 2016) to predict data in the following wave (e.g. 2017). We initially tested such a procedure using simulated data and then tested different models on various topics from the European Social Survey. Both toy models and empirical models perform well on the simulated data, but fail to predict future years in the empirical data. Furthermore, during the calibration phase on the empirical data, most models learned to "freeze"--meaning that their predictions for the following year are just a copy of the data from the previous year. This work advances the literature by offering a benchmark for comparing different opinion dynamics models. Furthermore, our tests show that real-world dynamics appear to be completely incompatible with the dynamics of the tested models. This calls for more effort in exploring what are the features that would improve validity and applications for opinion dynamics models.