VERITAS-NLI : Validation and Extraction of Reliable Information Through Automated Scraping and Natural Language Inference

TL;DR

This paper introduces VERITAS-NLI, a system combining web-scraping and Natural Language Inference to verify news headlines, significantly improving fake news detection accuracy over traditional ML and BERT models.

Contribution

The paper presents a novel approach that integrates web-scraping with NLI models for external knowledge retrieval to verify headlines, addressing generalization issues of existing models.

Findings

Achieved 84.3% accuracy in fake news detection

Outperformed classical ML models by 33.3%

Surpassed BERT by 31.0% in accuracy

Abstract

In today's day and age where information is rapidly spread through online platforms, the rise of fake news poses an alarming threat to the integrity of public discourse, societal trust, and reputed news sources. Classical machine learning and Transformer-based models have been extensively studied for the task of fake news detection, however they are hampered by their reliance on training data and are unable to generalize on unseen headlines. To address these challenges, we propose our novel solution, leveraging web-scraping techniques and Natural Language Inference (NLI) models to retrieve external knowledge necessary for verifying the accuracy of a headline. Our system is evaluated on a diverse self-curated evaluation dataset spanning over multiple news channels and broad domains. Our best performing pipeline achieves an accuracy of 84.3% surpassing the best classical Machine Learning model by 33.3% and Bidirectional Encoder Representations from Transformers (BERT) by 31.0% . This highlights the efficacy of combining dynamic web-scraping with Natural Language Inference to find support for a claimed headline in the corresponding externally retrieved knowledge for the task of fake news detection.