Leveraging Large Language Models for Information Verification -- an Engineering Approach

TL;DR

This paper presents an engineering pipeline that leverages Large Language Models like GPT-4o for multimedia news source verification, automating metadata generation, frame analysis, and discrepancy detection with minimal human intervention.

Contribution

It introduces a practical, automated approach using LLMs for multimedia verification, integrating multiple data modalities and prompt engineering to streamline the process.

Findings

Automated pipeline effectively verifies multimedia news sources.

Integration of LLMs improves efficiency and accuracy.

Minimal human intervention required for validation.

Abstract

For the ACMMM25 challenge, we present a practical engineering approach to multimedia news source verification, utilizing Large Language Models (LLMs) like GPT-4o as the backbone of our pipeline. Our method processes images and videos through a streamlined sequence of steps: First, we generate metadata using general-purpose queries via Google tools, capturing relevant content and links. Multimedia data is then segmented, cleaned, and converted into frames, from which we select the top-K most informative frames. These frames are cross-referenced with metadata to identify consensus or discrepancies. Additionally, audio transcripts are extracted for further verification. Noticeably, the entire pipeline is automated using GPT-4o through prompt engineering, with human intervention limited to final validation.