ClearMask: Noise-Free and Naturalness-Preserving Protection Against Voice Deepfake Attacks

TL;DR

ClearMask introduces a noise-free, frequency-based audio modification technique combined with style transfer and reverberation to defend against voice deepfake attacks, maintaining natural sound quality and working in real-time.

Contribution

It proposes a novel noise-free defense mechanism, ClearMask, that selectively filters frequencies and applies style transfer to prevent voice deepfakes without degrading audio quality.

Findings

Effective against unseen voice synthesis models

Resilient to adaptive attackers

Works in real-time for streaming speech

Abstract

Voice deepfake attacks, which artificially impersonate human speech for malicious purposes, have emerged as a severe threat. Existing defenses typically inject noise into human speech to compromise voice encoders in speech synthesis models. However, these methods degrade audio quality and require prior knowledge of the attack approaches, limiting their effectiveness in diverse scenarios. Moreover, real-time audios, such as speech in virtual meetings and voice messages, are still exposed to voice deepfake threats. To overcome these limitations, we propose ClearMask, a noise-free defense mechanism against voice deepfake attacks. Unlike traditional approaches, ClearMask modifies the audio mel-spectrogram by selectively filtering certain frequencies, inducing a transferable voice feature loss without injecting noise. We then apply audio style transfer to further deceive voice decoders while preserving perceived sound quality. Finally, optimized reverberation is introduced to disrupt the output of voice generation models without affecting the naturalness of the speech. Additionally, we develop LiveMask to protect streaming speech in real-time through a universal frequency filter and reverberation generator. Our experimental results show that ClearMask and LiveMask effectively prevent voice deepfake attacks from deceiving speaker verification models and human listeners, even for unseen voice synthesis models and black-box API services. Furthermore, ClearMask demonstrates resilience against adaptive attackers who attempt to recover the original audio signal from the protected speech samples.