E-CaTCH: Event-Centric Cross-Modal Attention with Temporal Consistency and Class-Imbalance Handling for Misinformation Detection

TL;DR

E-CaTCH is a scalable, interpretable framework that detects multimodal misinformation by modeling event-level structures, temporal evolution, and addressing class imbalance, outperforming existing methods across multiple datasets.

Contribution

The paper introduces E-CaTCH, a novel event-centric, cross-modal attention framework that captures temporal dynamics and handles class imbalance for misinformation detection.

Findings

E-CaTCH outperforms state-of-the-art baselines on Fakeddit, IND, and COVID-19 MISINFOGRAPH datasets.

The model demonstrates robustness and generalizability across diverse misinformation scenarios.

Incorporating temporal consistency and class imbalance handling improves detection accuracy.

Abstract

Detecting multimodal misinformation on social media remains challenging due to inconsistencies between modalities, changes in temporal patterns, and substantial class imbalance. Many existing methods treat posts independently and fail to capture the event-level structure that connects them across time and modality. We propose E-CaTCH, an interpretable and scalable framework for robustly detecting misinformation. If needed, E-CaTCH clusters posts into pseudo-events based on textual similarity and temporal proximity, then processes each event independently. Within each event, textual and visual features are extracted using pre-trained BERT and ResNet encoders, refined via intra-modal self-attention, and aligned through bidirectional cross-modal attention. A soft gating mechanism fuses these representations to form contextualized, content-aware embeddings of each post. To model temporal evolution, E-CaTCH segments events into overlapping time windows and uses a trend-aware LSTM, enhanced with semantic shift and momentum signals, to encode narrative progression over time. Classification is performed at the event level, enabling better alignment with real-world misinformation dynamics. To address class imbalance and promote stable learning, the model integrates adaptive class weighting, temporal consistency regularization, and hard-example mining. The total loss is aggregated across all events. Extensive experiments on Fakeddit, IND, and COVID-19 MISINFOGRAPH demonstrate that E-CaTCH consistently outperforms state-of-the-art baselines. Cross-dataset evaluations further demonstrate its robustness, generalizability, and practical applicability across diverse misinformation scenarios.