Zero-shot Singing Technique Conversion

TL;DR

This paper introduces a modified AutoVC neural network framework for zero-shot singing technique conversion, enabling the transformation of singing styles without prior training on specific techniques.

Contribution

It presents a novel approach using a pretrained technique encoder and discusses training modifications for improved singing technique conversion.

Findings

Omission of latent loss improves conversion quality

Sequential training enhances model performance

Fine-tuning the bottleneck improves naturalness and specificity

Abstract

In this paper we propose modifications to the neural network framework, AutoVC for the task of singing technique conversion. This includes utilising a pretrained singing technique encoder which extracts technique information, upon which a decoder is conditioned during training. By swapping out a source singer's technique information for that of the target's during conversion, the input spectrogram is reconstructed with the target's technique. We document the beneficial effects of omitting the latent loss, the importance of sequential training, and our process for fine-tuning the bottleneck. We also conducted a listening study where participants rate the specificity of technique-converted voices as well as their naturalness. From this we are able to conclude how effective the technique conversions are and how different conditions affect them, while assessing the model's ability to reconstruct its input data.