Rapidly Adapting to New Voice Spoofing: Few-Shot Detection of Synthesized Speech Under Distribution Shifts

TL;DR

This paper introduces a self-attentive prototypical network for few-shot detection of synthesized speech, enabling rapid adaptation to unseen speech synthesis methods, speakers, and languages under distribution shifts, with significant performance improvements.

Contribution

The paper proposes a novel self-attentive prototypical network that enhances few-shot adaptation for voice spoofing detection under distribution shifts, outperforming traditional zero-shot detectors.

Findings

Achieves up to 32% relative EER reduction on deepfakes in Japanese.

Achieves up to 20% relative EER reduction on ASVspoof 2021 Deepfake dataset.

Effective with as few as 10 in-distribution samples.

Abstract

We address the challenge of detecting synthesized speech under distribution shifts -- arising from unseen synthesis methods, speakers, languages, or audio conditions -- relative to the training data. Few-shot learning methods are a promising way to tackle distribution shifts by rapidly adapting on the basis of a few in-distribution samples. We propose a self-attentive prototypical network to enable more robust few-shot adaptation. To evaluate our approach, we systematically compare the performance of traditional zero-shot detectors and the proposed few-shot detectors, carefully controlling training conditions to introduce distribution shifts at evaluation time. In conditions where distribution shifts hamper the zero-shot performance, our proposed few-shot adaptation technique can quickly adapt using as few as 10 in-distribution samples -- achieving upto 32% relative EER reduction on deepfakes in Japanese language and 20% relative reduction on ASVspoof 2021 Deepfake dataset.