Beyond Hate: Differentiating Uncivil and Intolerant Speech in Multimodal Content Moderation

TL;DR

This paper introduces a fine-grained annotation scheme to distinguish incivility and intolerance in multimodal content, improving moderation accuracy and reducing harmful content under different model training approaches.

Contribution

It proposes a novel annotation scheme based on communication science, applied to memes, and demonstrates how combining coarse and fine-grained labels enhances moderation models.

Findings

Fine-grained annotations improve model performance.

Models trained with fine-grained labels have more balanced error profiles.

Combining coarse and fine labels enhances moderation reliability.

Abstract

Current multimodal toxicity benchmarks typically use a single binary hatefulness label. This coarse approach conflates two fundamentally different characteristics of expression: tone and content. Drawing on communication science theory, we introduce a fine-grained annotation scheme that distinguishes two separable dimensions: incivility (rude or dismissive tone) and intolerance (content that attacks pluralism and targets groups or identities) and apply it to 2,030 memes from the Hateful Memes dataset. We evaluate different vision-language models under coarse-label training, transfer learning across label schemes and a joint learning approach that combines the coarse hatefulness label with our fine-grained annotations. Our results show that fine-grained annotations complement existing coarse labels and, when used jointly, improve overall model performance. Moreover, models trained with the fine-grained scheme exhibit more balanced moderation-relevant error profiles and are less prone to under-detection of harmful content than models trained on hatefulness labels alone (FNR-FPR, the difference between false negative and false positive rates: 0.74 to 0.42 for LLaVA-1.6-Mistral-7B; 0.54 to 0.28 for Qwen2.5-VL-7B). This work contributes to data-centric approaches in content moderation by improving the reliability and accuracy of moderation systems through enhanced data quality. Overall, combining both coarse and fine-grained labels provides a practical route to more reliable multimodal moderation.