A Universal Music Translation Network
Noam Mor, Lior Wolf, Adam Polyak, Yaniv Taigman

TL;DR
This paper introduces a universal music translation network that can convert music across instruments, genres, and styles using an unsupervised, multi-domain autoencoder capable of translating even unseen musical domains.
Contribution
It presents a novel multi-domain wavenet autoencoder with a shared encoder and disentangled latent space, enabling unsupervised translation across diverse musical domains, including unseen ones.
Findings
Achieves convincing music translation on NSynth and professional musician datasets.
Enables translation from whistling, facilitating instrumental music creation by untrained users.
Operates without supervision or matched sample pairs.
Abstract
We present a method for translating music across musical instruments, genres, and styles. This method is based on a multi-domain wavenet autoencoder, with a shared encoder and a disentangled latent space that is trained end-to-end on waveforms. Employing a diverse training dataset and large net capacity, the domain-independent encoder allows us to translate even from musical domains that were not seen during training. The method is unsupervised and does not rely on supervision in the form of matched samples between domains or musical transcriptions. We evaluate our method on NSynth, as well as on a dataset collected from professional musicians, and achieve convincing translations, even when translating from whistling, potentially enabling the creation of instrumental music by untrained humans.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMusic and Audio Processing · Music Technology and Sound Studies · Diverse Musicological Studies
MethodsMixture of Logistic Distributions · Dilated Causal Convolution · WaveNet
