Out-of-Context Misinformation Detection via Variational Domain-Invariant Learning with Test-Time Training

TL;DR

This paper introduces VDT, a novel method that improves out-of-context misinformation detection by learning domain-invariant features and adapting dynamically during testing, especially effective in unseen news domains.

Contribution

The paper proposes a domain-invariant variational learning framework with test-time training for out-of-context misinformation detection, addressing domain shift issues.

Findings

Outperforms state-of-the-art baselines on NewsCLIPpings dataset.

Effective in adapting to unseen news domains.

Enhances detection accuracy under domain shift conditions.

Abstract

Out-of-context misinformation (OOC) is a low-cost form of misinformation in news reports, which refers to place authentic images into out-of-context or fabricated image-text pairings. This problem has attracted significant attention from researchers in recent years. Current methods focus on assessing image-text consistency or generating explanations. However, these approaches assume that the training and test data are drawn from the same distribution. When encountering novel news domains, models tend to perform poorly due to the lack of prior knowledge. To address this challenge, we propose \textbf{VDT} to enhance the domain adaptation capability for OOC misinformation detection by learning domain-invariant features and test-time training mechanisms. Domain-Invariant Variational Align module is employed to jointly encodes source and target domain data to learn a separable distributional space domain-invariant features. For preserving semantic integrity, we utilize domain consistency constraint module to reconstruct the source and target domain latent distribution. During testing phase, we adopt the test-time training strategy and confidence-variance filtering module to dynamically updating the VAE encoder and classifier, facilitating the model's adaptation to the target domain distribution. Extensive experiments conducted on the benchmark dataset NewsCLIPpings demonstrate that our method outperforms state-of-the-art baselines under most domain adaptation settings.