Many-to-Many Voice Conversion using Cycle-Consistent Variational Autoencoder with Multiple Decoders

TL;DR

This paper introduces a cycle-consistent variational autoencoder with multiple decoders for many-to-many voice conversion, improving sound quality without requiring parallel training data.

Contribution

It proposes a novel cycle consistency loss and multiple decoders to enhance VAE-based voice conversion quality using non-parallel data.

Findings

Improved sound quality in converted speech.

Effective cycle consistency loss enhances conversion accuracy.

Validated through objective and subjective evaluations.

Abstract

One of the obstacles in many-to-many voice conversion is the requirement of the parallel training data, which contain pairs of utterances with the same linguistic content spoken by different speakers. Since collecting such parallel data is a highly expensive task, many works attempted to use non-parallel training data for many-to-many voice conversion. One of such approaches is using the variational autoencoder (VAE). Though it can handle many-to-many voice conversion without the parallel training, the VAE based voice conversion methods suffer from low sound qualities of the converted speech. One of the major reasons is because the VAE learns only the self-reconstruction path. The conversion path is not trained at all. In this paper, we propose a cycle consistency loss for VAE to explicitly learn the conversion path. In addition, we propose to use multiple decoders to further improve the sound qualities of the conventional VAE based voice conversion methods. The effectiveness of the proposed method is validated using objective and the subjective evaluations.