DisC-VC: Disentangled and F0-Controllable Neural Voice Conversion

TL;DR

This paper introduces DisC-VC, a neural voice conversion model that effectively controls F0 pitch patterns while preserving speaker identity, improving naturalness and accuracy through a novel variational autoencoder approach.

Contribution

The paper proposes a new variational-autoencoder-based voice conversion model with an auxiliary network to better control F0 patterns and reduce discrepancies during training.

Findings

Improved F0 control accuracy in voice conversion.

Enhanced naturalness of converted speech.

Objective and subjective evaluations confirm effectiveness.

Abstract

Voice conversion is a task to convert a non-linguistic feature of a given utterance. Since naturalness of speech strongly depends on its pitch pattern, in some applications, it would be desirable to keep the original rise/fall pitch pattern while changing the speaker identity. Some of the existing methods address this problem by either using a source-filter model or developing a neural network that takes an F0 pattern as input to the model. Although the latter approach can achieve relatively high sound quality compared to the former one, there is no consideration for discrepancy between the target and generated F0 patterns in its training process. In this paper, we propose a new variational-autoencoder-based voice conversion model accompanied by an auxiliary network, which ensures that the conversion result correctly reflects the specified F0/timbre information. We show the effectiveness of the proposed method by objective and subjective evaluations.