RW-Post: Auditable Evidence-Grounded Multimodal Fact-Checking in the Wild

TL;DR

RW-Post is a new multimodal fact-checking benchmark with auditable annotations, enabling systematic evaluation of models' ability to ground evidence and verify visual and textual misinformation.

Contribution

It introduces RW-Post, a comprehensive benchmark with auditable evidence annotations and a pipeline for extracting and auditing evidence from social media posts.

Findings

Current models struggle with faithful evidence grounding.

Evidence-bounded evaluation improves accuracy and faithfulness.

Code and dataset will be publicly released.

Abstract

Multimodal misinformation increasingly leverages visual persuasion, where repurposed or manipulated images strengthen misleading text. We introduce \textbf{RW-Post}, a post-aligned \textbf{text--image benchmark} for real-world multimodal fact-checking with \emph{auditable} annotations: each instance links the original social-media post with reasoning traces and explicitly linked evidence items derived from human fact-check articles via an LLM-assisted extraction-and-auditing pipeline. RW-Post supports controlled evaluation across closed-book, evidence-bounded, and open-web regimes, enabling systematic diagnosis of visual grounding and evidence utilization. We provide \textbf{AgentFact} as a reference verification baseline and benchmark strong open-source LVLMs under unified protocols. Experiments show substantial headroom: current models struggle with faithful evidence grounding, while evidence-bounded evaluation improves both accuracy and faithfulness. Code and dataset will be released at https://github.com/xudanni0927/AgentFact.