Neural Vocoder Feature Estimation for Dry Singing Voice Separation

TL;DR

This paper introduces a novel singing voice separation method that predicts dry vocal mel-spectrograms using neural vocoder features, improving separation quality over existing models by focusing on dereverberation and reusability.

Contribution

It proposes predicting dry singing voice mel-spectrograms with neural vocoder features and incorporates a singing voice detector, advancing separation techniques beyond spectrogram masking.

Findings

Outperforms state-of-the-art models in objective metrics

Achieves better dereverberation and separation quality

Improves reusability of isolated singing voices

Abstract

Singing voice separation (SVS) is a task that separates singing voice audio from its mixture with instrumental audio. Previous SVS studies have mainly employed the spectrogram masking method which requires a large dimensionality in predicting the binary masks. In addition, they focused on extracting a vocal stem that retains the wet sound with the reverberation effect. This result may hinder the reusability of the isolated singing voice. This paper addresses the issues by predicting mel-spectrogram of dry singing voices from the mixed audio as neural vocoder features and synthesizing the singing voice waveforms from the neural vocoder. We experimented with two separation methods. One is predicting binary masks in the mel-spectrogram domain and the other is directly predicting the mel-spectrogram. Furthermore, we add a singing voice detector to identify the singing voice segments over time more explicitly. We measured the model performance in terms of audio, dereverberation, separation, and overall quality. The results show that our proposed model outperforms state-of-the-art singing voice separation models in both objective and subjective evaluation except the audio quality.