Online Speaker Adaptation for WaveNet-based Neural Vocoders

TL;DR

This paper introduces an online speaker adaptation method for WaveNet vocoders that uses a speaker encoder to improve waveform reconstruction for unseen speakers, enhancing both objective and subjective quality.

Contribution

The paper presents a novel online speaker adaptation approach using a speaker encoder and a speaker-aware WaveNet vocoder for improved speaker-independent speech synthesis.

Findings

Better waveform reconstruction for unseen speakers

Improved objective and subjective performance

Effective adaptation with a speaker encoder

Abstract

In this paper, we propose an online speaker adaptation method for WaveNet-based neural vocoders in order to improve their performance on speaker-independent waveform generation. In this method, a speaker encoder is first constructed using a large speaker-verification dataset which can extract a speaker embedding vector from an utterance pronounced by an arbitrary speaker. At the training stage, a speaker-aware WaveNet vocoder is then built using a multi-speaker dataset which adopts both acoustic feature sequences and speaker embedding vectors as conditions.At the generation stage, we first feed the acoustic feature sequence from a test speaker into the speaker encoder to obtain the speaker embedding vector of the utterance. Then, both the speaker embedding vector and acoustic features pass the speaker-aware WaveNet vocoder to reconstruct speech waveforms. Experimental results demonstrate that our method can achieve a better objective and subjective performance on reconstructing waveforms of unseen speakers than the conventional speaker-independent WaveNet vocoder.