CHEF: A Pilot Chinese Dataset for Evidence-Based Fact-Checking

TL;DR

This paper introduces CHEF, a new Chinese dataset with 10,000 claims and evidence annotations, to advance automated fact-checking in Chinese across multiple domains.

Contribution

It presents the first Chinese evidence-based fact-checking dataset and a novel end-to-end model for evidence retrieval and veracity prediction.

Findings

CHEF provides a challenging testbed for Chinese fact-checking systems.

The proposed model effectively jointly trains evidence retrieval and veracity prediction.

Experiments demonstrate the dataset's utility for developing non-English fact-checking tools.

Abstract

The explosion of misinformation spreading in the media ecosystem urges for automated fact-checking. While misinformation spans both geographic and linguistic boundaries, most work in the field has focused on English. Datasets and tools available in other languages, such as Chinese, are limited. In order to bridge this gap, we construct CHEF, the first CHinese Evidence-based Fact-checking dataset of 10K real-world claims. The dataset covers multiple domains, ranging from politics to public health, and provides annotated evidence retrieved from the Internet. Further, we develop established baselines and a novel approach that is able to model the evidence retrieval as a latent variable, allowing jointly training with the veracity prediction model in an end-to-end fashion. Extensive experiments show that CHEF will provide a challenging testbed for the development of fact-checking systems designed to retrieve and reason over non-English claims.