A Domain-adaptive Pre-training Approach for Language Bias Detection in News

TL;DR

This paper introduces DA-RoBERTa, a domain-adapted transformer model for detecting word choice bias in news, achieving state-of-the-art performance and outperforming previous methods on bias detection tasks.

Contribution

The paper presents a novel domain-adaptive pre-training approach for transformer models specifically targeting media bias detection in news articles.

Findings

DA-RoBERTa achieves an F1 score of 0.814 in sentence-level bias detection.

Domain-adapted models outperform prior bias detection approaches.

Proposes two additional domain-adapted models, DA-BERT and DA-BART.

Abstract

Media bias is a multi-faceted construct influencing individual behavior and collective decision-making. Slanted news reporting is the result of one-sided and polarized writing which can occur in various forms. In this work, we focus on an important form of media bias, i.e. bias by word choice. Detecting biased word choices is a challenging task due to its linguistic complexity and the lack of representative gold-standard corpora. We present DA-RoBERTa, a new state-of-the-art transformer-based model adapted to the media bias domain which identifies sentence-level bias with an F1 score of 0.814. In addition, we also train, DA-BERT and DA-BART, two more transformer models adapted to the bias domain. Our proposed domain-adapted models outperform prior bias detection approaches on the same data.