A Speech Representation Anonymization Framework via Selective Noise Perturbation

TL;DR

This paper introduces a novel speech anonymization framework that uses selective noise perturbation on high-utility speech representations, balancing privacy and utility without retraining components.

Contribution

It presents a new privacy-preserving speech anonymization method based on noise perturbation guided by a Transformer-based saliency estimator, outperforming existing baselines.

Findings

Achieves comparable or better utility than VoicePrivacy2022 baselines.

Provides flexible privacy-utility trade-offs without re-training.

Maintains privacy levels comparable to existing methods.

Abstract

Privacy and security are major concerns when communicating speech signals to cloud services such as automatic speech recognition (ASR) and speech emotion recognition (SER). Existing solutions for speech anonymization mainly focus on voice conversion or voice modification to convert a raw utterance into another one with similar content but different, or no, identity-related information. However, an alternative approach to share speech data under the form of privacy-preserving representation has been largely under-explored. In this paper, we propose a speech anonymization framework that achieves privacy via noise perturbation to a selected subset of the high-utility representations extracted using a pre-trained speech encoder. The subset is chosen with a Transformer-based privacy-risk saliency estimator. We validate our framework on four tasks, namely, Automatic Speaker Verification (ASV), ASR, SER and Intent Classification (IC) for privacy and utility assessment. Experimental results show that our approach is able to achieve a competitive, or even better, utility compared to the speech anonymization baselines from the VoicePrivacy2022 Challenges, providing the same level of privacy. Moreover, the easily-controlled amount of perturbation allows our framework to have a flexible range of privacy-utility trade-offs without re-training any component.