POLygraph: Polish Fake News Dataset

TL;DR

The paper introduces POLygraph, a comprehensive Polish fake news dataset with annotated news articles and comments, aiming to advance fake news detection in Polish through new data resources and tools.

Contribution

It presents the creation of a novel, multi-part Polish fake news dataset and a software tool for analyzing content authenticity, filling a gap in resources for Polish-language fake news detection.

Findings

Dataset includes 11,360 fake news pairs and 5,082 commented articles.

Manual annotation by experts and non-experts ensures data quality.

Provides a foundation for future fake news detection models in Polish.

Abstract

This paper presents the POLygraph dataset, a unique resource for fake news detection in Polish. The dataset, created by an interdisciplinary team, is composed of two parts: the "fake-or-not" dataset with 11,360 pairs of news articles (identified by their URLs) and corresponding labels, and the "fake-they-say" dataset with 5,082 news articles (identified by their URLs) and tweets commenting on them. Unlike existing datasets, POLygraph encompasses a variety of approaches from source literature, providing a comprehensive resource for fake news detection. The data was collected through manual annotation by expert and non-expert annotators. The project also developed a software tool that uses advanced machine learning techniques to analyze the data and determine content authenticity. The tool and dataset are expected to benefit various entities, from public sector institutions to publishers and fact-checking organizations. Further dataset exploration will foster fake news detection and potentially stimulate the implementation of similar models in other languages. The paper focuses on the creation and composition of the dataset, so it does not include a detailed evaluation of the software tool for content authenticity analysis, which is planned at a later stage of the project.