Towards Evaluating the Robustness of Automatic Speech Recognition Systems via Audio Style Transfer

TL;DR

This paper introduces a novel style transfer-based attack on ASR systems that allows user customization, achieving high success rates while maintaining audio naturalness, addressing limitations of previous adversarial methods.

Contribution

The paper proposes a new attack method combining style transfer and adversarial techniques, improving controllability and naturalness in attacking ASR systems.

Findings

Achieves 82% attack success rate.

Maintains sound naturalness according to user studies.

Enables user-customized audio styles in adversarial attacks.

Abstract

In light of the widespread application of Automatic Speech Recognition (ASR) systems, their security concerns have received much more attention than ever before, primarily due to the susceptibility of Deep Neural Networks. Previous studies have illustrated that surreptitiously crafting adversarial perturbations enables the manipulation of speech recognition systems, resulting in the production of malicious commands. These attack methods mostly require adding noise perturbations under $\ell_p$ norm constraints, inevitably leaving behind artifacts of manual modifications. Recent research has alleviated this limitation by manipulating style vectors to synthesize adversarial examples based on Text-to-Speech (TTS) synthesis audio. However, style modifications based on optimization objectives significantly reduce the controllability and editability of audio styles. In this paper, we propose an attack on ASR systems based on user-customized style transfer. We first test the effect of Style Transfer Attack (STA) which combines style transfer and adversarial attack in sequential order. And then, as an improvement, we propose an iterative Style Code Attack (SCA) to maintain audio quality. Experimental results show that our method can meet the need for user-customized styles and achieve a success rate of 82% in attacks, while keeping sound naturalness due to our user study.