Improving the Speaker Anonymization Evaluation's Robustness to Target Speakers with Adversarial Learning

TL;DR

This paper enhances speaker anonymization evaluation by incorporating adversarial learning to better account for target speaker influence, especially under same-gender target selection, resulting in more accurate privacy assessments.

Contribution

It introduces a target classifier and adversarial training to improve the robustness of speaker anonymization evaluation against gender-based vulnerabilities.

Findings

The approach effectively reduces target speaker information in anonymized speech.

It provides a more reliable privacy evaluation for anonymization methods.

The method works well across multiple anonymizers, especially with same-gender TSA.

Abstract

The current privacy evaluation for speaker anonymization often overestimates privacy when a same-gender target selection algorithm (TSA) is used, although this TSA leaks the speaker's gender and should hence be more vulnerable. We hypothesize that this occurs because the evaluation does not account for the fact that anonymized speech contains information from both the source and target speakers. To address this, we propose to add a target classifier that measures the influence of target speaker information in the evaluation, which can also be removed with adversarial learning. Experiments demonstrate that this approach is effective for multiple anonymizers, particularly when using a same-gender TSA, leading to a more reliable assessment.