End-to-End Speaker Diarization Conditioned on Speech Activity and Overlap Detection

TL;DR

This paper introduces a multitask learning framework for end-to-end speaker diarization that explicitly models speech activity and overlap detection, leading to improved diarization accuracy over existing methods.

Contribution

The paper proposes a novel multitask learning approach that conditions speaker diarization on speech activity and overlap detection, enhancing performance compared to traditional EEND systems.

Findings

Outperforms conventional EEND in diarization error rate

Effectively models speaker diarization using subtasks

Leverages subtask information to improve diarization accuracy

Abstract

In this paper, we present a conditional multitask learning method for end-to-end neural speaker diarization (EEND). The EEND system has shown promising performance compared with traditional clustering-based methods, especially in the case of overlapping speech. In this paper, to further improve the performance of the EEND system, we propose a novel multitask learning framework that solves speaker diarization and a desired subtask while explicitly considering the task dependency. We optimize speaker diarization conditioned on speech activity and overlap detection that are subtasks of speaker diarization, based on the probabilistic chain rule. Experimental results show that our proposed method can leverage a subtask to effectively model speaker diarization, and outperforms conventional EEND systems in terms of diarization error rate.