Testing the Robustness of a BiLSTM-based Structural Story Classifier

TL;DR

This paper evaluates how noise, such as mislabeling, affects the performance of a BiLSTM-based structural model for fake news detection, highlighting the importance of robustness in machine learning classifiers.

Contribution

It provides an analysis of the robustness of a state-of-the-art BiLSTM-based fake news classifier against noisy data, an area not extensively studied before.

Findings

Noise significantly degrades classifier performance

Robustness varies with noise levels

Highlights need for noise-resistant models

Abstract

The growing prevalence of counterfeit stories on the internet has fostered significant interest towards fast and scalable detection of fake news in the machine learning community. While several machine learning techniques for this purpose have emerged, we observe that there is a need to evaluate the impact of noise on these techniques' performance, where noise constitutes news articles being mistakenly labeled as fake (or real). This work takes a step in that direction, where we examine the impact of noise on a state-of-the-art, structural model based on BiLSTM (Bidirectional Long-Short Term Model) for fake news detection, Hierarchical Discourse-level Structure for Fake News Detection by Karimi and Tang (Reference no. 9).