Mitigation of Diachronic Bias in Fake News Detection Dataset

TL;DR

This paper identifies diachronic bias in fake news datasets caused by temporal dependence on proper nouns and proposes Wikidata-based masking methods to improve model robustness against such biases.

Contribution

It introduces a novel approach using Wikidata to mask proper nouns, mitigating diachronic bias in fake news detection datasets.

Findings

Masking proper nouns reduces bias in datasets.

Proposed methods improve detection robustness.

Experiments show effectiveness on in-domain and out-of-domain data.

Abstract

Fake news causes significant damage to society.To deal with these fake news, several studies on building detection models and arranging datasets have been conducted. Most of the fake news datasets depend on a specific time period. Consequently, the detection models trained on such a dataset have difficulty detecting novel fake news generated by political changes and social changes; they may possibly result in biased output from the input, including specific person names and organizational names. We refer to this problem as \textbf{Diachronic Bias} because it is caused by the creation date of news in each dataset. In this study, we confirm the bias, especially proper nouns including person names, from the deviation of phrase appearances in each dataset. Based on these findings, we propose masking methods using Wikidata to mitigate the influence of person names and validate whether they make fake news detection models robust through experiments with in-domain and out-of-domain data.