On Investigation of Unsupervised Speech Factorization Based on Normalization Flow

TL;DR

This paper explores an unsupervised speech factorization method using normalization flow models, demonstrating that speech can be decomposed into meaningful factors like phonetic content and speaker traits within a latent space.

Contribution

It introduces a novel application of normalization flow for speech factorization, revealing properties of the latent space that facilitate disentangling speech attributes.

Findings

Latent space exhibits denseness and pseudo linearity.

Phonetic content and speaker traits are represented as specific directions.

Preliminary results on TIMIT show promising factorization properties.

Abstract

Speech signals are complex composites of various information, including phonetic content, speaker traits, channel effect, etc. Decomposing this complicated mixture into independent factors, i.e., speech factorization, is fundamentally important and plays the central role in many important algorithms of modern speech processing tasks. In this paper, we present a preliminary investigation on unsupervised speech factorization based on the normalization flow model. This model constructs a complex invertible transform, by which we can project speech segments into a latent code space where the distribution is a simple diagonal Gaussian. Our preliminary investigation on the TIMIT database shows that this code space exhibits favorable properties such as denseness and pseudo linearity, and perceptually important factors such as phonetic content and speaker trait can be represented as particular directions within the code space.