Exposing AI-Synthesized Human Voices Using Neural Vocoder Artifacts

TL;DR

This paper presents a novel detection method for AI-synthesized human voices by identifying neural vocoder artifacts using a multi-task learning framework, significantly improving classification accuracy.

Contribution

It introduces a multi-task learning approach that leverages vocoder artifact detection to enhance synthetic voice detection performance.

Findings

High classification accuracy achieved with the proposed model.

Vocoder artifact detection improves synthetic voice discrimination.

Multi-task learning constrains feature extraction for better results.

Abstract

The advancements of AI-synthesized human voices have introduced a growing threat of impersonation and disinformation. It is therefore of practical importance to developdetection methods for synthetic human voices. This work proposes a new approach to detect synthetic human voices based on identifying artifacts of neural vocoders in audio signals. A neural vocoder is a specially designed neural network that synthesizes waveforms from temporal-frequency representations, e.g., mel-spectrograms. The neural vocoder is a core component in most DeepFake audio synthesis models. Hence the identification of neural vocoder processing implies that an audio sample may have been synthesized. To take advantage of the vocoder artifacts for synthetic human voice detection, we introduce a multi-task learning framework for a binary-class RawNet2 model that shares the front-end feature extractor with a vocoder identification module. We treat the vocoder identification as a pretext task to constrain the front-end feature extractor to focus on vocoder artifacts and provide discriminative features for the final binary classifier. Our experiments show that the improved RawNet2 model based on vocoder identification achieves an overall high classification performance on the binary task.