Multimodal Fake News Video Explanation: Dataset, Analysis and Evaluation

TL;DR

This paper introduces a new dataset and method for explaining fake news videos by generating natural language explanations, addressing the interpretability challenge in multimodal fake news detection.

Contribution

It presents FakeVE, a novel dataset for fake news video explanation, and proposes the MRGT model to benchmark and analyze explanation generation in multimodal fake news videos.

Findings

FakeVE dataset contains 2,672 annotated fake news videos.

Benchmark models show convincing results in explanation generation.

Analysis reveals key differences in explanation quality across models.

Abstract

Multimodal fake news videos are difficult to interpret because they require comprehensive consideration of the correlation and consistency between multiple modes. Existing methods deal with fake news videos as a classification problem, but it's not clear why news videos are identified as fake. Without proper explanation, the end user may not understand the underlying meaning of the falsehood. Therefore, we propose a new problem - Fake news video Explanation (FNVE) - given a multimodal news post containing a video and title, our goal is to generate natural language explanations to reveal the falsity of the news video. To that end, we developed FakeVE, a new dataset of 2,672 fake news video posts that can definitively explain four real-life fake news video aspects. In order to understand the characteristics of fake news video explanation, we conducted an exploratory analysis of FakeVE from different perspectives. In addition, we propose a Multimodal Relation Graph Transformer (MRGT) based on the architecture of multimodal Transformer to benchmark FakeVE. The empirical results show that the results of the various benchmarks (adopted by FakeVE) are convincing and provide a detailed analysis of the differences in explanation generation of the benchmark models.