Zero-shot Singing Technique Conversion
Brendan O'Connor, Simon Dixon, George Fazekas

TL;DR
This paper introduces a modified AutoVC neural network framework for zero-shot singing technique conversion, enabling the transformation of singing styles without prior training on specific techniques.
Contribution
It presents a novel approach using a pretrained technique encoder and discusses training modifications for improved singing technique conversion.
Findings
Omission of latent loss improves conversion quality
Sequential training enhances model performance
Fine-tuning the bottleneck improves naturalness and specificity
Abstract
In this paper we propose modifications to the neural network framework, AutoVC for the task of singing technique conversion. This includes utilising a pretrained singing technique encoder which extracts technique information, upon which a decoder is conditioned during training. By swapping out a source singer's technique information for that of the target's during conversion, the input spectrogram is reconstructed with the target's technique. We document the beneficial effects of omitting the latent loss, the importance of sequential training, and our process for fine-tuning the bottleneck. We also conducted a listening study where participants rate the specificity of technique-converted voices as well as their naturalness. From this we are able to conclude how effective the technique conversions are and how different conditions affect them, while assessing the model's ability to…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMusic and Audio Processing · Speech and Audio Processing · Music Technology and Sound Studies
