Inference of Media Bias and Content Quality Using Natural-Language Processing

TL;DR

This paper introduces a neural network-based framework to quantitatively assess media bias and content quality from text, demonstrating superior performance over traditional methods using real-world Twitter data.

Contribution

It presents a novel LSTM-based approach for inferring media bias and content quality, outperforming baseline bag-of-words machine learning methods.

Findings

LSTM approach outperforms baseline methods in accuracy

Word order information improves bias and quality inference

Framework applied successfully to over 1 million tweets

Abstract

Media bias can significantly impact the formation and development of opinions and sentiments in a population. It is thus important to study the emergence and development of partisan media and political polarization. However, it is challenging to quantitatively infer the ideological positions of media outlets. In this paper, we present a quantitative framework to infer both political bias and content quality of media outlets from text, and we illustrate this framework with empirical experiments with real-world data. We apply a bidirectional long short-term memory (LSTM) neural network to a data set of more than 1 million tweets to generate a two-dimensional ideological-bias and content-quality measurement for each tweet. We then infer a ``media-bias chart'' of (bias, quality) coordinates for the media outlets by integrating the (bias, quality) measurements of the tweets of the media outlets. We also apply a variety of baseline machine-learning methods, such as a naive-Bayes method and a support-vector machine (SVM), to infer the bias and quality values for each tweet. All of these baseline approaches are based on a bag-of-words approach. We find that the LSTM-network approach has the best performance of the examined methods. Our results illustrate the importance of leveraging word order into machine-learning methods in text analysis.