Privacy-preserving Automatic Speaker Diarization

TL;DR

This paper introduces a privacy-preserving automatic speaker diarization system that leverages cryptographic techniques to protect user privacy during voice data processing, achieving real-time performance.

Contribution

It is the first to combine Secure Multiparty Computation and Secure Modular Hashing for privacy-preserving ASD, addressing a previously overlooked area.

Findings

Achieves real-time processing with factors of 1.1 and 1.6.

Balances privacy and performance effectively.

Introduces a novel cryptographic approach to ASD.

Abstract

Automatic Speaker Diarization (ASD) is an enabling technology with numerous applications, which deals with recordings of multiple speakers, raising special concerns in terms of privacy. In fact, in remote settings, where recordings are shared with a server, clients relinquish not only the privacy of their conversation, but also of all the information that can be inferred from their voices. However, to the best of our knowledge, the development of privacy-preserving ASD systems has been overlooked thus far. In this work, we tackle this problem using a combination of two cryptographic techniques, Secure Multiparty Computation (SMC) and Secure Modular Hashing, and apply them to the two main steps of a cascaded ASD system: speaker embedding extraction and agglomerative hierarchical clustering. Our system is able to achieve a reasonable trade-off between performance and efficiency, presenting real-time factors of 1.1 and 1.6, for two different SMC security settings.