Revealing Temporal Label Noise in Multimodal Hateful Video Classification

TL;DR

This paper investigates how coarse video-level annotations introduce label noise in multimodal hateful video detection, revealing the importance of temporal granularity for accurate classification and model robustness.

Contribution

It provides a fine-grained analysis of temporal label noise in hateful videos and demonstrates its impact on model decision boundaries and confidence, emphasizing the need for temporally aware models.

Findings

Temporal label noise affects model decisions and confidence.

Fine-grained analysis reveals semantic overlap in hateful segments.

Temporal context is crucial for robust hate speech detection.

Abstract

The rapid proliferation of online multimedia content has intensified the spread of hate speech, presenting critical societal and regulatory challenges. While recent work has advanced multimodal hateful video detection, most approaches rely on coarse, video-level annotations that overlook the temporal granularity of hateful content. This introduces substantial label noise, as videos annotated as hateful often contain long non-hateful segments. In this paper, we investigate the impact of such label ambiguity through a fine-grained approach. Specifically, we trim hateful videos from the HateMM and MultiHateClip English datasets using annotated timestamps to isolate explicitly hateful segments. We then conduct an exploratory analysis of these trimmed segments to examine the distribution and characteristics of both hateful and non-hateful content. This analysis highlights the degree of semantic overlap and the confusion introduced by coarse, video-level annotations. Finally, controlled experiments demonstrated that time-stamp noise fundamentally alters model decision boundaries and weakens classification confidence, highlighting the inherent context dependency and temporal continuity of hate speech expression. Our findings provide new insights into the temporal dynamics of multimodal hateful videos and highlight the need for temporally aware models and benchmarks for improved robustness and interpretability. Code and data are available at https://github.com/Multimodal-Intelligence-Lab-MIL/HatefulVideoLabelNoise.