A Benchmark for Multi-speaker Anonymization

TL;DR

This paper introduces a benchmark for multi-speaker voice anonymization, addressing privacy concerns in real-world scenarios by proposing a system that anonymizes overlapping conversations while maintaining speech utility.

Contribution

It defines a new multi-speaker anonymization benchmark, proposes a cascaded system with novel speaker anonymization methods, and analyzes privacy leakage in overlapping speech.

Findings

Effective anonymization of multi-speaker conversations demonstrated

Proposed methods maintain speaker relationships and improve differentiation

Analysis of privacy leakage in overlapping speech scenarios

Abstract

Privacy-preserving voice protection approaches primarily suppress privacy-related information derived from paralinguistic attributes while preserving the linguistic content. Existing solutions focus particularly on single-speaker scenarios. However, they lack practicality for real-world applications, i.e., multi-speaker scenarios. In this paper, we present an initial attempt to provide a multi-speaker anonymization benchmark by defining the task and evaluation protocol, proposing benchmarking solutions, and discussing the privacy leakage of overlapping conversations. The proposed benchmark solutions are based on a cascaded system that integrates spectral-clustering-based speaker diarization and disentanglement-based speaker anonymization using a selection-based anonymizer. To improve utility, the benchmark solutions are further enhanced by two conversation-level speaker vector anonymization methods. The first method minimizes the differential similarity across speaker pairs in the original and anonymized conversations, which maintains original speaker relationships in the anonymized version. The other minimizes the aggregated similarity across anonymized speakers, which achieves better differentiation between speakers.Experiments conducted on both non-overlap simulated and real-world datasets demonstrate the effectiveness of the multi-speaker anonymization system with the proposed speaker anonymizers. Additionally, we analyzed overlapping speech regarding privacy leakage and provided potential solutions