Lexical Squad@Multimodal Hate Speech Event Detection 2023: Multimodal Hate Speech Detection using Fused Ensemble Approach

TL;DR

This paper introduces a novel ensemble approach combining InceptionV3, BERT, and XLNet for detecting hate speech in multimodal social media content, achieving promising accuracy and F-1 scores.

Contribution

The paper presents a new multimodal ensemble learning method for hate speech detection using fused models, which is a novel approach in this domain.

Findings

Achieved 75.21% accuracy in hate speech detection.

Attained 74.96 F-1 score, demonstrating effective classification.

Evaluated multimodal content to understand model performance.

Abstract

With a surge in the usage of social media postings to express opinions, emotions, and ideologies, there has been a significant shift towards the calibration of social media as a rapid medium of conveying viewpoints and outlooks over the globe. Concurrently, the emergence of a multitude of conflicts between two entities has given rise to a stream of social media content containing propaganda, hate speech, and inconsiderate views. Thus, the issue of monitoring social media postings is rising swiftly, attracting major attention from those willing to solve such problems. One such problem is Hate Speech detection. To mitigate this problem, we present our novel ensemble learning approach for detecting hate speech, by classifying text-embedded images into two labels, namely "Hate Speech" and "No Hate Speech". We have incorporated state-of-art models including InceptionV3, BERT, and XLNet. Our proposed ensemble model yielded promising results with 75.21 and 74.96 as accuracy and F-1 score (respectively). We also present an empirical evaluation of the text-embedded images to elaborate on how well the model was able to predict and classify. We release our codebase here (https://github.com/M0hammad-Kashif/MultiModalHateSpeech).