Independent fact-checking organizations exhibit a departure from political neutrality

TL;DR

This study develops a longitudinal measure of political neutrality for fact-checking organizations, revealing subtle biases in their reporting that influence public perception in the USA and India from 2018 to 2023.

Contribution

It introduces a novel, cross-national measure of political neutrality for fact-checkers, extending beyond traditional left-right bias assessments.

Findings

Average neutrality scores of -0.17 in the USA and -0.24 in India.

Fact-checking organizations exhibit subtle political biases.

Biases impact reader perceptions of political entities.

Abstract

Independent fact-checking organizations have emerged as the crusaders to debunk fake news. However, they may not always remain neutral, as they can be selective in the false news they choose to expose and in how they present the information. They can deviate from neutrality by being selective in what false news they debunk and how the information is presented. Prompting the now popular large language model, GPT-3.5, with journalistic frameworks, we establish a longitudinal measure (2018-2023) for political neutrality that looks beyond the left-right spectrum. Specified on a range of -1 to 1 (with zero being absolute neutrality), we establish the extent of negative portrayal of political entities that makes a difference in the readers' perception in the USA and India. Here, we observe an average score of -0.17 and -0.24 in the USA and India, respectively. The findings indicate how seemingly objective fact-checking can still carry distorted political views, indirectly and subtly impacting the perception of consumers of the news.