MDD: a Mask Diffusion Detector to Protect Speaker Verification Systems from Adversarial Perturbations

TL;DR

This paper introduces MDD, a diffusion-based framework that detects and purifies adversarial perturbations in speaker verification systems without needing adversarial examples or extensive pretraining.

Contribution

MDD is a novel diffusion model-based detector that improves adversarial detection and purification in speaker verification, outperforming existing methods without relying on adversarial data.

Findings

MDD achieves superior adversarial detection accuracy.

MDD effectively restores speaker verification performance.

The method does not require adversarial examples or large-scale pretraining.

Abstract

Speaker verification systems are increasingly deployed in security-sensitive applications but remain highly vulnerable to adversarial perturbations. In this work, we propose the Mask Diffusion Detector (MDD), a novel adversarial detection and purification framework based on a \textit{text-conditioned masked diffusion model}. During training, MDD applies partial masking to Mel-spectrograms and progressively adds noise through a forward diffusion process, simulating the degradation of clean speech features. A reverse process then reconstructs the clean representation conditioned on the input transcription. Unlike prior approaches, MDD does not require adversarial examples or large-scale pretraining. Experimental results show that MDD achieves strong adversarial detection performance and outperforms prior state-of-the-art methods, including both diffusion-based and neural codec-based approaches. Furthermore, MDD effectively purifies adversarially-manipulated speech, restoring speaker verification performance to levels close to those observed under clean conditions. These findings demonstrate the potential of diffusion-based masking strategies for secure and reliable speaker verification systems.