RMDM: A Multilabel Fakenews Dataset for Vietnamese Evidence Verification

TL;DR

This paper introduces RMDM, a challenging Vietnamese multilabel dataset for fake news verification in legal contexts, highlighting the difficulty language models face in authenticating electronic evidence.

Contribution

The study presents a novel Vietnamese multilabel fake news dataset (RMDM) with diverse labels, designed to evaluate and improve language models' ability to verify electronic information in legal scenarios.

Findings

GPT-based models show varied performance across labels.

BERT-based models also exhibit inconsistent accuracy.

Fake news verification remains a challenging task for current language models.

Abstract

In this study, we present a novel and challenging multilabel Vietnamese dataset (RMDM) designed to assess the performance of large language models (LLMs), in verifying electronic information related to legal contexts, focusing on fake news as potential input for electronic evidence. The RMDM dataset comprises four labels: real, mis, dis, and mal, representing real information, misinformation, disinformation, and mal-information, respectively. By including these diverse labels, RMDM captures the complexities of differing fake news categories and offers insights into the abilities of different language models to handle various types of information that could be part of electronic evidence. The dataset consists of a total of 1,556 samples, with 389 samples for each label. Preliminary tests on the dataset using GPT-based and BERT-based models reveal variations in the models' performance across different labels, indicating that the dataset effectively challenges the ability of various language models to verify the authenticity of such information. Our findings suggest that verifying electronic information related to legal contexts, including fake news, remains a difficult problem for language models, warranting further attention from the research community to advance toward more reliable AI models for potential legal applications.