ViWikiFC: Fact-Checking for Vietnamese Wikipedia-Based Textual Knowledge Source

TL;DR

This paper introduces ViWikiFC, the first annotated Vietnamese Wikipedia fact-checking corpus, and evaluates various models for evidence retrieval and verdict prediction, highlighting challenges in low-resource language fact-checking.

Contribution

The creation of ViWikiFC, a novel Vietnamese fact-checking dataset, and comprehensive experiments demonstrating the challenges and potential of current models in low-resource language fact verification.

Findings

BM25 achieved 88.30% accuracy in evidence retrieval.

InfoXLM (Large) achieved an F1 score of 86.51%.

Pipeline approach had 67.00% strict accuracy.

Abstract

Fact-checking is essential due to the explosion of misinformation in the media ecosystem. Although false information exists in every language and country, most research to solve the problem mainly concentrated on huge communities like English and Chinese. Low-resource languages like Vietnamese are necessary to explore corpora and models for fact verification. To bridge this gap, we construct ViWikiFC, the first manual annotated open-domain corpus for Vietnamese Wikipedia Fact Checking more than 20K claims generated by converting evidence sentences extracted from Wikipedia articles. We analyze our corpus through many linguistic aspects, from the new dependency rate, the new n-gram rate, and the new word rate. We conducted various experiments for Vietnamese fact-checking, including evidence retrieval and verdict prediction. BM25 and InfoXLM (Large) achieved the best results in two tasks, with BM25 achieving an accuracy of 88.30% for SUPPORTS, 86.93% for REFUTES, and only 56.67% for the NEI label in the evidence retrieval task, InfoXLM (Large) achieved an F1 score of 86.51%. Furthermore, we also conducted a pipeline approach, which only achieved a strict accuracy of 67.00% when using InfoXLM (Large) and BM25. These results demonstrate that our dataset is challenging for the Vietnamese language model in fact-checking tasks.