TL;DR
This paper introduces BABE, a high-quality expert-annotated dataset for media bias detection at the word and sentence levels, and develops a BERT-based model that effectively identifies bias in news articles.
Contribution
The creation of BABE, a diverse and expert-annotated dataset for media bias, and a novel BERT-based approach that leverages distant supervision for improved bias detection.
Findings
BABE dataset has higher inter-annotator agreement than previous datasets.
The BERT-based model achieves a macro F1-score of 0.804 on bias detection.
Distant supervision enhances model performance in media bias identification.
Abstract
Media coverage has a substantial effect on the public perception of events. Nevertheless, media outlets are often biased. One way to bias news articles is by altering the word choice. The automatic identification of bias by word choice is challenging, primarily due to the lack of a gold standard data set and high context dependencies. This paper presents BABE, a robust and diverse data set created by trained experts, for media bias research. We also analyze why expert labeling is essential within this domain. Our data set offers better annotation quality and higher inter-annotator agreement than existing work. It consists of 3,700 sentences balanced among topics and outlets, containing media bias labels on the word and sentence level. Based on our data, we also introduce a way to detect bias-inducing sentences in news articles automatically. Our best performing BERT-based model is…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
