MEVER: Multi-Modal and Explainable Claim Verification with Graph-based Evidence Retrieval
Delvin Ce Zhang, Suhan Cui, Zhelin Chu, Xianren Zhang, Dongwon Lee

TL;DR
This paper introduces MEVER, a multi-modal, explainable claim verification model that retrieves evidence, verifies claims, and generates explanations using a graph-based approach, and also provides a new scientific dataset for AI claims.
Contribution
The paper presents a novel multi-modal, explainable claim verification framework with a graph-based evidence retrieval method and introduces a new scientific dataset for AI claim verification.
Findings
Effective multi-modal evidence retrieval via graph reasoning.
Improved claim verification accuracy with token- and evidence-level fusion.
Successful explanation generation with multi-modal Fusion-in-Decoder.
Abstract
Verifying the truthfulness of claims usually requires joint multi-modal reasoning over both textual and visual evidence, such as analyzing both textual caption and chart image for claim verification. In addition, to make the reasoning process transparent, a textual explanation is necessary to justify the verification result. However, most claim verification works mainly focus on the reasoning over textual evidence only or ignore the explainability, resulting in inaccurate and unconvincing verification. To address this problem, we propose a novel model that jointly achieves evidence retrieval, multi-modal claim verification, and explanation generation. For evidence retrieval, we construct a two-layer multi-modal graph for claims and evidence, where we design image-to-text and text-to-image reasoning for multi-modal retrieval. For claim verification, we propose token- and evidence-level…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
Taxonomy
TopicsMultimodal Machine Learning Applications · Topic Modeling · Explainable Artificial Intelligence (XAI)
