MultiCheck: Strengthening Web Trust with Unified Multimodal Fact Verification

TL;DR

MultiCheck is an efficient, transparent multimodal fact verification framework that effectively detects misinformation across text, images, and OCR content, suitable for low-resource environments.

Contribution

It introduces a novel relational fusion module and contrastive alignment objective for multimodal fact verification, emphasizing interpretability and robustness.

Findings

Significant performance improvements on Factify-2 and Mocheg benchmarks.

Robustness under noisy OCR and missing modality conditions.

Low computational overhead suitable for resource-constrained deployment.

Abstract

Misinformation on the web increasingly appears in multimodal forms, combining text, images, and OCR-rendered content in ways that amplify harm to public trust and vulnerable communities. While prior fact-checking systems often rely on unimodal signals or shallow fusion strategies, modern misinformation campaigns operate across modalities and require models that can reason over subtle cross-modal inconsistencies in a transparent and responsible manner. We introduce MultiCheck, a lightweight and interpretable framework for multimodal fact verification that jointly analyzes textual, visual, and OCR evidence. At its core, MultiCheck employs a relational fusion module based on element-wise difference and product operations, allowing for explicit cross-modal interaction modeling with minimal computational overhead. A contrastive alignment objective further helps the model distinguish between supporting and refuting evidence while maintaining a small memory and energy footprint, making it suitable for low-resource deployment. Evaluated on the Factify-2 (5-class) and Mocheg (3-class) benchmarks, MultiCheck achieves huge performance improvement and remains robust under noisy OCR and missing modality conditions. Its efficiency, transparency, and real-world robustness make it well-suited for journalists, civil society organisations, and web integrity efforts working to build a safer and more trustworthy web.