Perception, Understanding and Reasoning, A Multimodal Benchmark for Video Fake News Detection

TL;DR

This paper introduces POVFNDB, a comprehensive, process-oriented benchmark with 36,240 QA pairs across 15 dimensions, to evaluate and improve multimodal large language models' capabilities in video fake news detection.

Contribution

It presents a new benchmark and evaluation framework that assesses perception, understanding, and reasoning in VFND, along with a strong baseline model achieving state-of-the-art results.

Findings

POVFNDB enables detailed assessment of MLLMs in VFND.

Fine-tuning Qwen2.5VL-7B-Instruct with POVFND-CoT achieves state-of-the-art performance.

The benchmark covers 15 evaluation dimensions for comprehensive analysis.

Abstract

The advent of multi-modal large language models (MLLMs) has greatly advanced research on video fake news detection (VFND) tasks. Existing benchmarks typically focus on the detection accuracy, while failing to provide fine-grained assessments for the entire detection process. To address these limitations, we introduce {POVFNDB (Process-oriented Video Fake News Detection Benchmark)}, a process-oriented benchmark comprising 10 tasks designed to systematically evaluate MLLMs' perception, understanding, and reasoning capabilities in VFND. This benchmark contains \textit{36,240} human-annotated question-answer (QA) in structured or open-ended formats, spanning 15 distinct evaluation dimensions that characterize different aspects of the video fake news detection process. Using POVFNDB, we conduct comprehensive evaluations on both proprietary and open-source MLLMs. Moreover, we establish a strong benchmark baseline by fine-tuning Qwen2.5VL-7B-Instruct on process-oriented chain-of-thought data constructed with our proposed POVFND-CoT framework, achieving state-of-the-art performance on VFND.