SynHate: Detecting Hate Speech in Synthetic Deepfake Audio

TL;DR

SynHate introduces a multilingual dataset for detecting hate speech in synthetic deepfake audio, highlighting challenges in cross-language generalization and providing a benchmark for future research.

Contribution

The paper presents SynHate, the first multilingual dataset for hate speech detection in synthetic audio, and evaluates multiple models to establish baseline performance.

Findings

Whisper-small performs best overall among evaluated models.

Performance varies significantly across different languages.

Cross-dataset generalization remains a key challenge.

Abstract

The rise of deepfake audio and hate speech, powered by advanced text-to-speech, threatens online safety. We present SynHate, the first multilingual dataset for detecting hate speech in synthetic audio, spanning 37 languages. SynHate uses a novel four-class scheme: Real-normal, Real-hate, Fake-normal, and Fake-hate. Built from MuTox and ADIMA datasets, it captures diverse hate speech patterns globally and in India. We evaluate five leading self-supervised models (Whisper-small/medium, XLS-R, AST, mHuBERT), finding notable performance differences by language, with Whisper-small performing best overall. Cross-dataset generalization remains a challenge. By releasing SynHate and baseline code, we aim to advance robust, culturally sensitive, and multilingual solutions against synthetic hate speech. The dataset is available at https://www.iab-rubric.org/resources.