MIMIC: Multimodal Islamophobic Meme Identification and Classification

TL;DR

This paper introduces a new multimodal dataset and a ViLT-based classifier to detect Islamophobic hate speech in memes by combining visual and textual analysis for improved accuracy.

Contribution

It presents a novel dataset and a multimodal transformer model specifically designed for identifying Islamophobic content in memes, addressing the challenge of context-dependent hate speech detection.

Findings

High detection accuracy achieved

Effective integration of visual and textual features

Model captures nuanced Islamophobic narratives

Abstract

Anti-Muslim hate speech has emerged within memes, characterized by context-dependent and rhetorical messages using text and images that seemingly mimic humor but convey Islamophobic sentiments. This work presents a novel dataset and proposes a classifier based on the Vision-and-Language Transformer (ViLT) specifically tailored to identify anti-Muslim hate within memes by integrating both visual and textual representations. Our model leverages joint modal embeddings between meme images and incorporated text to capture nuanced Islamophobic narratives that are unique to meme culture, providing both high detection accuracy and interoperability.