Improving Zero-shot Voice Style Transfer via Disentangled Representation Learning

TL;DR

This paper introduces a novel zero-shot voice style transfer method that uses disentangled representation learning to effectively transfer voice styles to unseen speakers, achieving state-of-the-art results.

Contribution

It proposes a disentangled representation learning approach that encodes style and content separately, enabling zero-shot voice transfer with improved accuracy and naturalness.

Findings

Outperforms baseline methods on VCTK dataset

Achieves state-of-the-art transfer accuracy

Enhances voice naturalness in zero-shot transfer

Abstract

Voice style transfer, also called voice conversion, seeks to modify one speaker's voice to generate speech as if it came from another (target) speaker. Previous works have made progress on voice conversion with parallel training data and pre-known speakers. However, zero-shot voice style transfer, which learns from non-parallel data and generates voices for previously unseen speakers, remains a challenging problem. We propose a novel zero-shot voice transfer method via disentangled representation learning. The proposed method first encodes speaker-related style and voice content of each input voice into separated low-dimensional embedding spaces, and then transfers to a new voice by combining the source content embedding and target style embedding through a decoder. With information-theoretic guidance, the style and content embedding spaces are representative and (ideally) independent of each other. On real-world VCTK datasets, our method outperforms other baselines and obtains state-of-the-art results in terms of transfer accuracy and voice naturalness for voice style transfer experiments under both many-to-many and zero-shot setups.