Simultaneous Speech Extraction for Multiple Target Speakers under the Meeting Scenarios

TL;DR

This paper introduces a novel multi-target speech separation model that simultaneously extracts multiple speakers' voices in meeting scenarios, improving speech quality and diarization accuracy without needing prior enrollment audio.

Contribution

It proposes the MTSS model for concurrent multi-speaker extraction and the SD-MTSS system integrating speaker diarization with speech separation, addressing inter-speaker relationships.

Findings

Achieved 1.38dB SDR improvement over baseline

Improved 0.13 PESQ score on WSJ0-2mix-extr dataset

Reduced 19.2% CER on Alimeeting dataset

Abstract

The common target speech separation directly estimate the target source, ignoring the interrelationship between different speakers at each frame. We propose a multiple-target speech separation model (MTSS) to simultaneously extract each speaker's voice from the mixed speech rather than just optimally estimating the target source. Moreover, we propose a speaker diarization (SD) aware MTSS system (SD-MTSS), which consists of a SD module and MTSS module. By exploiting the TSVAD decision and the estimated mask, our SD-MTSS model can extract the speech signal of each speaker concurrently in a conversational recording without additional enrollment audio in advance. Experimental results show that our MTSS model achieves 1.38dB SDR, 1.34dB SI-SDR, and 0.13 PESQ improvements over the baseline on the WSJ0-2mix-extr dataset, respectively. The SD-MTSS system makes 19.2% relative speaker dependent character error rate (CER) reduction on the Alimeeting dataset.