Changes in European Solidarity Before and During COVID-19: Evidence from a Large Crowd- and Expert-Annotated Twitter Dataset

TL;DR

This study develops a machine learning approach to analyze how European solidarity and anti-solidarity discourses on Twitter evolved before and during COVID-19, revealing increased salience and contestation of solidarity during the pandemic.

Contribution

It introduces a novel NLP framework combining expert and crowd annotations to accurately classify (anti-)solidarity expressions in tweets, applied to large-scale data over time.

Findings

Solidarity discourse increased and became more contested during COVID-19.

Anti-solidarity tweets spiked then decreased before rising again.

The augmented BERT model achieved over 85% macro-F1 in classification.

Abstract

We introduce the well-established social scientific concept of social solidarity and its contestation, anti-solidarity, as a new problem setting to supervised machine learning in NLP to assess how European solidarity discourses changed before and after the COVID-19 outbreak was declared a global pandemic. To this end, we annotate 2.3k English and German tweets for (anti-)solidarity expressions, utilizing multiple human annotators and two annotation approaches (experts vs.\ crowds). We use these annotations to train a BERT model with multiple data augmentation strategies. Our augmented BERT model that combines both expert and crowd annotations outperforms the baseline BERT classifier trained with expert annotations only by over 25 points, from 58\% macro-F1 to almost 85\%. We use this high-quality model to automatically label over 270k tweets between September 2019 and December 2020. We then assess the automatically labeled data for how statements related to European (anti-)solidarity discourses developed over time and in relation to one another, before and during the COVID-19 crisis. Our results show that solidarity became increasingly salient and contested during the crisis. While the number of solidarity tweets remained on a higher level and dominated the discourse in the scrutinized time frame, anti-solidarity tweets initially spiked, then decreased to (almost) pre-COVID-19 values before rising to a stable higher level until the end of 2020.