Three-class Overlapped Speech Detection using a Convolutional Recurrent Neural Network

TL;DR

This paper introduces a three-class overlapped speech detection system using a convolutional recurrent neural network, achieving state-of-the-art performance and improving speaker diarization accuracy.

Contribution

The work presents a novel three-class classification approach for overlapped speech detection and demonstrates its effectiveness with a convolutional recurrent neural network architecture.

Findings

State-of-the-art precision of 0.6648 on DIHARD II

Recall improved by 20% over previous methods

Third place in DIHARD III speaker diarization challenge

Abstract

In this work, we propose an overlapped speech detection system trained as a three-class classifier. Unlike conventional systems that perform binary classification as to whether or not a frame contains overlapped speech, the proposed approach classifies into three classes: non-speech, single speaker speech, and overlapped speech. By training a network with the more detailed label definition, the model can learn a better notion on deciding the number of speakers included in a given frame. A convolutional recurrent neural network architecture is explored to benefit from both convolutional layer's capability to model local patterns and recurrent layer's ability to model sequential information. The proposed overlapped speech detection model establishes a state-of-the-art performance with a precision of 0.6648 and a recall of 0.3222 on the DIHARD II evaluation set, showing a 20% increase in recall along with higher precision. In addition, we also introduce a simple approach to utilize the proposed overlapped speech detection model for speaker diarization which ranked third place in the Track 1 of the DIHARD III challenge.