Reverb Conversion of Mixed Vocal Tracks Using an End-to-end Convolutional Deep Neural Network

TL;DR

This paper introduces an end-to-end convolutional neural network system that can transfer or remove reverb effects between vocal tracks, enabling more precise control over spatial and acoustic characteristics in music production.

Contribution

It presents the first deep learning approach for converting reverb in vocal tracks, capable of both applying and removing reverb effects using an adversarial training framework.

Findings

Perceptual evaluation shows 64.8% preferred reverb conversion rate.

The model effectively applies reverb from reference tracks to source vocals.

It can also perform de-reverberation when needed.

Abstract

Reverb plays a critical role in music production, where it provides listeners with spatial realization, timbre, and texture of the music. Yet, it is challenging to reproduce the musical reverb of a reference music track even by skilled engineers. In response, we propose an end-to-end system capable of switching the musical reverb factor of two different mixed vocal tracks. This method enables us to apply the reverb of the reference track to the source track to which the effect is desired. Further, our model can perform de-reverberation when the reference track is used as a dry vocal source. The proposed model is trained in combination with an adversarial objective, which makes it possible to handle high-resolution audio samples. The perceptual evaluation confirmed that the proposed model can convert the reverb factor with the preferred rate of 64.8%. To the best of our knowledge, this is the first attempt to apply deep neural networks to converting music reverb of vocal tracks.