PopBERT. Detecting populism and its host ideologies in the German Bundestag

TL;DR

This paper introduces PopBERT, a transformer-based model trained on a new annotated dataset of German Bundestag speeches, to reliably detect and analyze populist language and its ideological attachments, enabling dynamic political discourse analysis.

Contribution

It presents a novel annotated dataset and a transformer-based classifier for detecting populist language and its ideological context in German parliamentary speeches.

Findings

PopBERT achieves high predictive accuracy.

The model aligns with expert party rankings.

It correctly detects out-of-sample populist statements.

Abstract

The rise of populism concerns many political scientists and practitioners, yet the detection of its underlying language remains fragmentary. This paper aims to provide a reliable, valid, and scalable approach to measure populist stances. For that purpose, we created an annotated dataset based on parliamentary speeches of the German Bundestag (2013 to 2021). Following the ideational definition of populism, we label moralizing references to the virtuous people or the corrupt elite as core dimensions of populist language. To identify, in addition, how the thin ideology of populism is thickened, we annotate how populist statements are attached to left-wing or right-wing host ideologies. We then train a transformer-based model (PopBERT) as a multilabel classifier to detect and quantify each dimension. A battery of validation checks reveals that the model has a strong predictive accuracy, provides high qualitative face validity, matches party rankings of expert surveys, and detects out-of-sample text snippets correctly. PopBERT enables dynamic analyses of how German-speaking politicians and parties use populist language as a strategic device. Furthermore, the annotator-level data may also be applied in cross-domain applications or to develop related classifiers.