MultiHateLoc: Towards Temporal Localisation of Multimodal Hate Content in Online Videos

TL;DR

This paper introduces MultiHateLoc, a novel weakly-supervised framework for localising multimodal hate speech in online videos, effectively capturing temporal and cross-modal dynamics to produce fine-grained, interpretable predictions.

Contribution

MultiHateLoc is the first framework to address weakly-supervised temporal localisation of multimodal hate speech, integrating modality-aware encoders, dynamic fusion, and contrastive alignment.

Findings

Achieves state-of-the-art localisation performance on HateMM and MultiHateClip datasets.

Effectively models heterogeneous temporal patterns across modalities.

Produces fine-grained, interpretable frame-level predictions.

Abstract

The rapid growth of video content on platforms such as TikTok and YouTube has intensified the spread of multimodal hate speech, where harmful cues emerge subtly and asynchronously across visual, acoustic, and textual streams. Existing research primarily focuses on video-level classification, leaving the practically crucial task of temporal localisation, identifying when hateful segments occur, largely unaddressed. This challenge is even more noticeable under weak supervision, where only video-level labels are available, and static fusion or classification-based architectures struggle to capture cross-modal and temporal dynamics. To address these challenges, we propose MultiHateLoc, the first framework designed for weakly-supervised multimodal hate localisation. MultiHateLoc incorporates (1) modality-aware temporal encoders to model heterogeneous sequential patterns, including a tailored text-based preprocessing module for feature enhancement; (2) dynamic cross-modal fusion to adaptively emphasise the most informative modality at each moment and a cross-modal contrastive alignment strategy to enhance multimodal feature consistency; (3) a modality-aware MIL objective to identify discriminative segments under video-level supervision. Despite relying solely on coarse labels, MultiHateLoc produces fine-grained, interpretable frame-level predictions. Experiments on HateMM and MultiHateClip show that our method achieves state-of-the-art performance in the localisation task.