Deep generative factorization for speech signal

TL;DR

This paper introduces a factorial discriminative normalization flow model for speech signal factorization, effectively separating phonetic content and speaker traits, and demonstrating superior performance over existing models.

Contribution

The paper proposes a novel factorial DNF model that enhances speech signal factorization by effectively disentangling multiple information factors.

Findings

The factorial DNF outperforms comparative models in speech factorization tasks.

The model effectively separates phonetic content and speaker traits.

Experiments validate the model's capability to manipulate individual speech information factors.

Abstract

Various information factors are blended in speech signals, which forms the primary difficulty for most speech information processing tasks. An intuitive idea is to factorize speech signal into individual information factors (e.g., phonetic content and speaker trait), though it turns out to be highly challenging. This paper presents a speech factorization approach based on a novel factorial discriminative normalization flow model (factorial DNF). Experiments conducted on a two-factor case that involves phonetic content and speaker trait demonstrates that the proposed factorial DNF has powerful capability to factorize speech signals and outperforms several comparative models in terms of information representation and manipulation.