Hostility Detection in UK Politics: A Dataset on Online Abuse Targeting MPs

TL;DR

This paper introduces a new dataset of 3,320 annotated UK political tweets to analyze hostility and targeted identity in online abuse towards MPs, evaluating language models for detection tasks.

Contribution

It provides the first UK-specific dataset on political hostility with detailed annotations and performs linguistic analyses and model evaluations for hostility detection.

Findings

Pre-trained language models achieve moderate accuracy in hostility detection.

Targeted identity classification reveals distinct linguistic patterns.

UK-specific political hostility data enhances understanding of online abuse.

Abstract

Numerous politicians use social media platforms, particularly X, to engage with their constituents. This interaction allows constituents to pose questions and offer feedback but also exposes politicians to a barrage of hostile responses, especially given the anonymity afforded by social media. They are typically targeted in relation to their governmental role, but the comments also tend to attack their personal identity. This can discredit politicians and reduce public trust in the government. It can also incite anger and disrespect, leading to offline harm and violence. While numerous models exist for detecting hostility in general, they lack the specificity required for political contexts. Furthermore, addressing hostility towards politicians demands tailored approaches due to the distinct language and issues inherent to each country (e.g., Brexit for the UK). To bridge this gap, we construct a dataset of 3,320 English tweets spanning a two-year period manually annotated for hostility towards UK MPs. Our dataset also captures the targeted identity characteristics (race, gender, religion, none) in hostile tweets. We perform linguistic and topical analyses to delve into the unique content of the UK political data. Finally, we evaluate the performance of pre-trained language models and large language models on binary hostility detection and multi-class targeted identity type classification tasks. Our study offers valuable data and insights for future research on the prevalence and nature of politics-related hostility specific to the UK.