SPBA: Utilizing Speech Large Language Model for Backdoor Attacks on Speech Classification Models

TL;DR

This paper introduces SPBA, a novel backdoor attack on speech classification models utilizing speech prompts generated by a large language model, demonstrating high effectiveness and diversity of triggers.

Contribution

The paper presents SPBA, a new speech backdoor attack leveraging speech elements and large language models to generate diverse triggers, enhancing attack success.

Findings

SPBA achieves high attack success rates on speech classification tasks.

Diverse triggers increase poisoning effectiveness and attack complexity.

The proposed method outperforms existing backdoor techniques in speech models.

Abstract

Deep speech classification tasks, including keyword spotting and speaker verification, are vital in speech-based human-computer interaction. Recently, the security of these technologies has been revealed to be susceptible to backdoor attacks. Specifically, attackers use noisy disruption triggers and speech element triggers to produce poisoned speech samples that train models to become vulnerable. However, these methods typically create only a limited number of backdoors due to the inherent constraints of the trigger function. In this paper, we propose that speech backdoor attacks can strategically focus on speech elements such as timbre and emotion, leveraging the Speech Large Language Model (SLLM) to generate diverse triggers. Increasing the number of triggers may disproportionately elevate the poisoning rate, resulting in higher attack costs and a lower success rate per trigger. We introduce the Multiple Gradient Descent Algorithm (MGDA) as a mitigation strategy to address this challenge. The proposed attack is called the Speech Prompt Backdoor Attack (SPBA). Building on this foundation, we conducted attack experiments on two speech classification tasks, demonstrating that SPBA shows significant trigger effectiveness and achieves exceptional performance in attack metrics.