Multimodal Climate Disinformation Detection: Integrating Vision-Language Models with External Knowledge Sources

TL;DR

This paper presents a novel approach that combines vision-language models with external knowledge sources to improve detection of climate disinformation in images and videos, addressing limitations of existing models.

Contribution

It introduces a method integrating external real-time information with VLMs to enhance accuracy in identifying climate-related false claims.

Findings

Improved detection accuracy of climate disinformation.

Effective integration of external knowledge sources.

Enhanced reasoning about recent events.

Abstract

Climate disinformation has become a major challenge in today digital world, especially with the rise of misleading images and videos shared widely on social media. These false claims are often convincing and difficult to detect, which can delay actions on climate change. While vision-language models (VLMs) have been used to identify visual disinformation, they rely only on the knowledge available at the time of training. This limits their ability to reason about recent events or updates. The main goal of this paper is to overcome that limitation by combining VLMs with external knowledge. By retrieving up-to-date information such as reverse image results, online fact-checks, and trusted expert content, the system can better assess whether an image and its claim are accurate, misleading, false, or unverifiable. This approach improves the model ability to handle real-world climate disinformation and supports efforts to protect public understanding of science in a rapidly changing information landscape.