Voice Conversion Based Speaker Normalization for Acoustic Unit Discovery

TL;DR

This paper introduces an unsupervised, multilingual speaker normalization method using adversarial contrastive predictive coding to improve acoustic unit discovery without requiring transcriptions or speaker labels.

Contribution

It presents a novel unsupervised speaker normalization technique that enhances acoustic unit discovery across multiple languages without labeled data.

Findings

Improved acoustic unit discovery performance on English, Yoruba, and Mboshi.

Effective speaker normalization achieved even with limited target language data.

Method is compatible with various unit discovery systems.

Abstract

Discovering speaker independent acoustic units purely from spoken input is known to be a hard problem. In this work we propose an unsupervised speaker normalization technique prior to unit discovery. It is based on separating speaker related from content induced variations in a speech signal with an adversarial contrastive predictive coding approach. This technique does neither require transcribed speech nor speaker labels, and, furthermore, can be trained in a multilingual fashion, thus achieving speaker normalization even if only few unlabeled data is available from the target language. The speaker normalization is done by mapping all utterances to a medoid style which is representative for the whole database. We demonstrate the effectiveness of the approach by conducting acoustic unit discovery with a hidden Markov model variational autoencoder noting, however, that the proposed speaker normalization can serve as a front end to any unit discovery system. Experiments on English, Yoruba and Mboshi show improvements compared to using non-normalized input.