Salient Speech Representations Based on Cloned Networks

TL;DR

This paper introduces a method to extract salient, shared features from equivalent signals using cloned encoder networks, improving speech coding and enhancement with a WaveNet system.

Contribution

It proposes a novel joint training approach for encoder clones to identify shared salient features, enhancing generative speech models.

Findings

Effective extraction of salient speech features for conditioning

Improved speech coding and enhancement performance

Demonstrated applicability with WaveNet system

Abstract

We define salient features as features that are shared by signals that are defined as being equivalent by a system designer. The definition allows the designer to contribute qualitative information. We aim to find salient features that are useful as conditioning for generative networks. We extract salient features by jointly training a set of clones of an encoder network. Each network clone receives as input a different signal from a set of equivalent signals. The objective function encourages the network clones to map their input into a set of features that is identical across the clones. It additionally encourages feature independence and, optionally, reconstruction of a desired target signal by a decoder. As an application, we train a system that extracts a time-sequence of feature vectors of speech and uses it as a conditioning of a WaveNet generative system, facilitating both coding and enhancement.