Song Data Cleansing for End-to-End Neural Singer Diarization Using Neural Analysis and Synthesis Framework

TL;DR

This paper introduces a neural data cleansing method using NANSY++ to improve end-to-end neural singer diarization by converting choral singing into solo singing data, significantly reducing diarization errors.

Contribution

The novel approach leverages NANSY++ for data cleansing, enabling effective training of singer diarization models on choral-rich datasets, which was challenging before.

Findings

Achieved a 14.8 point reduction in diarization error rate.

Effectively converts choral singing to solo singing data for training.

Improves diarization performance on popular duet songs.

Abstract

We propose a data cleansing method that utilizes a neural analysis and synthesis (NANSY++) framework to train an end-to-end neural diarization model (EEND) for singer diarization. Our proposed model converts song data with choral singing which is commonly contained in popular music and unsuitable for generating a simulated dataset to the solo singing data. This cleansing is based on NANSY++, which is a framework trained to reconstruct an input non-overlapped audio signal. We exploit the pre-trained NANSY++ to convert choral singing into clean, non-overlapped audio. This cleansing process mitigates the mislabeling of choral singing to solo singing and helps the effective training of EEND models even when the majority of available song data contains choral singing sections. We experimentally evaluated the EEND model trained with a dataset using our proposed method using annotated popular duet songs. As a result, our proposed method improved 14.8 points in diarization error rate.