Multi-modal Hate Speech Detection using Machine Learning

TL;DR

This paper proposes a multimodal machine learning approach to detect hate speech in videos by combining features from images, audio, and text, addressing limitations of single-modality models.

Contribution

It introduces a novel multimodal system that integrates visual, audio, and textual features for more accurate hate speech detection in videos.

Findings

Improved detection accuracy over single-modality models

Effective feature extraction from images, audio, and text

Demonstrated feasibility of multimodal hate speech detection

Abstract

With the continuous growth of internet users and media content, it is very hard to track down hateful speech in audio and video. Converting video or audio into text does not detect hate speech accurately as human sometimes uses hateful words as humorous or pleasant in sense and also uses different voice tones or show different action in the video. The state-ofthe-art hate speech detection models were mostly developed on a single modality. In this research, a combined approach of multimodal system has been proposed to detect hate speech from video contents by extracting feature images, feature values extracted from the audio, text and used machine learning and Natural language processing.