Unsupervised Cross-Domain Singing Voice Conversion

TL;DR

This paper introduces an unsupervised, real-time, waveform-based model for cross-domain singing voice conversion that effectively transforms singing voices across different identities without requiring labeled data.

Contribution

It proposes a novel end-to-end convolutional architecture that uses acoustic and melody features to convert singing voices without supervision or parallel data.

Findings

Outperforms baseline methods in quality

Generates convincing singing voice conversions

Operates in real-time

Abstract

We present a wav-to-wav generative model for the task of singing voice conversion from any identity. Our method utilizes both an acoustic model, trained for the task of automatic speech recognition, together with melody extracted features to drive a waveform-based generator. The proposed generative architecture is invariant to the speaker's identity and can be trained to generate target singers from unlabeled training data, using either speech or singing sources. The model is optimized in an end-to-end fashion without any manual supervision, such as lyrics, musical notes or parallel samples. The proposed approach is fully-convolutional and can generate audio in real-time. Experiments show that our method significantly outperforms the baseline methods while generating convincingly better audio samples than alternative attempts.