Transfer Learning for Improving Singing-voice Detection in Polyphonic Instrumental Music

TL;DR

This paper introduces a transfer learning-based data augmentation method to improve singing-voice detection in polyphonic music, addressing the scarcity of labeled data and reducing domain mismatch.

Contribution

It proposes a novel transfer learning approach that enhances singing-voice detection accuracy by leveraging artificial data and small real datasets.

Findings

F-score improved from 89.5% to 93.2%.

Artificial data combined with transfer learning enhances detection accuracy.

Method reduces the need for extensive frame-level labeling.

Abstract

Detecting singing-voice in polyphonic instrumental music is critical to music information retrieval. To train a robust vocal detector, a large dataset marked with vocal or non-vocal label at frame-level is essential. However, frame-level labeling is time-consuming and labor expensive, resulting there is little well-labeled dataset available for singing-voice detection (S-VD). Hence, we propose a data augmentation method for S-VD by transfer learning. In this study, clean speech clips with voice activity endpoints and separate instrumental music clips are artificially added together to simulate polyphonic vocals to train a vocal/non-vocal detector. Due to the different articulation and phonation between speaking and singing, the vocal detector trained with the artificial dataset does not match well with the polyphonic music which is singing vocals together with the instrumental accompaniments. To reduce this mismatch, transfer learning is used to transfer the knowledge learned from the artificial speech-plus-music training set to a small but matched polyphonic dataset, i.e., singing vocals with accompaniments. By transferring the related knowledge to make up for the lack of well-labeled training data in S-VD, the proposed data augmentation method by transfer learning can improve S-VD performance with an F-score improvement from 89.5% to 93.2%.