Vocoder drift in x-vector-based speaker anonymization

TL;DR

This paper reveals that vocoder drift significantly affects speaker anonymization, can be learned and reversed, thus weakening privacy protection and highlighting the need to consider vocoder impacts in anonymization strategies.

Contribution

It demonstrates that vocoder drift is a learnable and reversible factor that can undermine speaker anonymization, emphasizing the importance of addressing vocoder effects.

Findings

Vocoder drift is substantial and sometimes dominant in anonymization.

Vocoder drift can be learned and reversed by an attacker.

Current focus on x-vector anonymization may be insufficient for privacy protection.

Abstract

State-of-the-art approaches to speaker anonymization typically employ some form of perturbation function to conceal speaker information contained within an x-vector embedding, then resynthesize utterances in the voice of a new pseudo-speaker using a vocoder. Strategies to improve the x-vector anonymization function have attracted considerable research effort, whereas vocoder impacts are generally neglected. In this paper, we show that the impact of the vocoder is substantial and sometimes dominant. The vocoder drift, namely the difference between the x-vector vocoder input and that which can be extracted subsequently from the output, is learnable and can hence be reversed by an attacker; anonymization can be undone and the level of privacy protection provided by such approaches might be weaker than previously thought. The findings call into question the focus upon x-vector anonymization, prompting the need for greater attention to vocoder impacts and stronger attack models alike.