Detecting Hateful Memes Using a Multimodal Deep Ensemble

TL;DR

This paper improves multimodal deep learning models for detecting hateful memes, achieving state-of-the-art performance and ranking fifth among over three thousand participants.

Contribution

It introduces enhancements to visual-linguistic Transformer architectures for hate speech detection, significantly boosting accuracy.

Findings

Model outperforms baseline methods by a large margin

Achieves 5th place on the leaderboard among 3,100+ teams

Demonstrates the effectiveness of proposed improvements

Abstract

While significant progress has been made using machine learning algorithms to detect hate speech, important technical challenges still remain to be solved in order to bring their performance closer to human accuracy. We investigate several of the most recent visual-linguistic Transformer architectures and propose improvements to increase their performance for this task. The proposed model outperforms the baselines by a large margin and ranks 5$^{th}$ on the leaderboard out of 3,100+ participants.