Fake the Real: Backdoor Attack on Deep Speech Classification via Voice Conversion

TL;DR

This paper introduces a novel backdoor attack on deep speech classification that uses voice conversion to create sample-specific triggers, avoiding audible noise and demonstrating high effectiveness and resistance to fine-tuning.

Contribution

It presents a new voice conversion-based backdoor attack that is sample-specific and stealthy, expanding the security threat landscape for speech classification models.

Findings

Effective attack demonstrated on two speech tasks

Triggers are unnoticeable and sample-specific

Resistant to fine-tuning defenses

Abstract

Deep speech classification has achieved tremendous success and greatly promoted the emergence of many real-world applications. However, backdoor attacks present a new security threat to it, particularly with untrustworthy third-party platforms, as pre-defined triggers set by the attacker can activate the backdoor. Most of the triggers in existing speech backdoor attacks are sample-agnostic, and even if the triggers are designed to be unnoticeable, they can still be audible. This work explores a backdoor attack that utilizes sample-specific triggers based on voice conversion. Specifically, we adopt a pre-trained voice conversion model to generate the trigger, ensuring that the poisoned samples does not introduce any additional audible noise. Extensive experiments on two speech classification tasks demonstrate the effectiveness of our attack. Furthermore, we analyzed the specific scenarios that activated the proposed backdoor and verified its resistance against fine-tuning.