A Deep-Bayesian Framework for Adaptive Speech Duration Modification

TL;DR

This paper introduces a Bayesian deep learning framework that adaptively modifies speech duration by estimating an attention map and alignment, achieving results comparable to dynamic time warping and state-of-the-art vocoders.

Contribution

It presents the first adaptive speech duration modification method using a Bayesian approach with a neural attention mechanism.

Findings

Achieves speech alignment similar to dynamic time warping.

Produces high-quality speech comparable to state-of-the-art vocoders.

Effective in voice and emotion conversion tasks.

Abstract

We propose the first method to adaptively modify the duration of a given speech signal. Our approach uses a Bayesian framework to define a latent attention map that links frames of the input and target utterances. We train a masked convolutional encoder-decoder network to produce this attention map via a stochastic version of the mean absolute error loss function; our model also predicts the length of the target speech signal using the encoder embeddings. The predicted length determines the number of steps for the decoder operation. During inference, we generate the attention map as a proxy for the similarity matrix between the given input speech and an unknown target speech signal. Using this similarity matrix, we compute a warping path of alignment between the two signals. Our experiments demonstrate that this adaptive framework produces similar results to dynamic time warping, which relies on a known target signal, on both voice conversion and emotion conversion tasks. We also show that our technique results in a high quality of generated speech that is on par with state-of-the-art vocoders.