Trustworthy Hate Speech Detection Through Visual Augmentation

TL;DR

This paper introduces TrusV-HSD, a novel multimodal approach that enhances hate speech detection by integrating visual data and a trustworthy loss to improve semantic understanding and reduce uncertainty.

Contribution

It presents a new multimodal hate speech detection method that leverages visual augmentation and trustworthy loss without needing paired data, improving detection accuracy.

Findings

Significant performance improvements over existing methods.

Effective extraction of trustworthy semantic information.

Robustness to uncertainty in hate speech detection.

Abstract

The surge of hate speech on social media platforms poses a significant challenge, with hate speech detection~(HSD) becoming increasingly critical. Current HSD methods focus on enriching contextual information to enhance detection performance, but they overlook the inherent uncertainty of hate speech. We propose a novel HSD method, named trustworthy hate speech detection method through visual augmentation (TrusV-HSD), which enhances semantic information through integration with diffused visual images and mitigates uncertainty with trustworthy loss. TrusV-HSD learns semantic representations by effectively extracting trustworthy information through multi-modal connections without paired data. Our experiments on public HSD datasets demonstrate the effectiveness of TrusV-HSD, showing remarkable improvements over conventional methods.