Universal Adaptor: Converting Mel-Spectrograms Between Different Configurations for Speech Synthesis

TL;DR

This paper introduces Universal Adaptor, a method to convert Mel-spectrograms between different configurations, enabling flexible integration of synthesizers and vocoders in speech synthesis systems without quality loss.

Contribution

We propose a universal adaptor that converts Mel-spectrograms across configurations, facilitating compatibility among diverse speech synthesis components.

Findings

Speech quality comparable to ground truth Mel-spectrograms

Effective in single- and multi-speaker scenarios

Applicable to TTS and voice conversion systems

Abstract

Most recent speech synthesis systems are composed of a synthesizer and a vocoder. However, the existing synthesizers and vocoders can only be matched to acoustic features extracted with a specific configuration. Hence, we can't combine arbitrary synthesizers and vocoders together to form a complete system, not to mention apply to a newly developed model. In this paper, we proposed Universal Adaptor, which takes a Mel-spectrogram parametrized by the source configuration and converts it into a Mel-spectrogram parametrized by the target configuration, as long as we feed in the source and the target configurations. Experiments show that the quality of speeches synthesized from our output of Universal Adaptor is comparable to those synthesized from ground truth Mel-spectrogram no matter in single-speaker or multi-speaker scenarios. Moreover, Universal Adaptor can be applied in the recent TTS systems and voice conversion systems without dropping quality.