DRVC: A Framework of Any-to-Any Voice Conversion with Self-Supervised Learning

TL;DR

This paper introduces DRVC, a self-supervised, end-to-end voice conversion framework that improves the disentanglement of content and speaker style, enhancing speech quality and similarity in any-to-any voice conversion tasks.

Contribution

The paper proposes a novel cycle-based disentanglement method within an end-to-end self-supervised model for more effective voice conversion.

Findings

Improved speech quality in converted voices.

Enhanced voice similarity to target speakers.

Effective disentanglement of content and style.

Abstract

Any-to-any voice conversion problem aims to convert voices for source and target speakers, which are out of the training data. Previous works wildly utilize the disentangle-based models. The disentangle-based model assumes the speech consists of content and speaker style information and aims to untangle them to change the style information for conversion. Previous works focus on reducing the dimension of speech to get the content information. But the size is hard to determine to lead to the untangle overlapping problem. We propose the Disentangled Representation Voice Conversion (DRVC) model to address the issue. DRVC model is an end-to-end self-supervised model consisting of the content encoder, timbre encoder, and generator. Instead of the previous work for reducing speech size to get content, we propose a cycle for restricting the disentanglement by the Cycle Reconstruct Loss and Same Loss. The experiments show there is an improvement for converted speech on quality and voice similarity.