Speaker Tagging Correction With Non-Autoregressive Language Models

TL;DR

This paper introduces a non-autoregressive language model-based system to correct speaker tagging errors in speech applications, significantly reducing diarization errors and improving overall speaker attribution accuracy.

Contribution

It presents a novel second-pass correction approach that enhances speaker tagging accuracy by addressing errors at sentence boundaries in speech recognition systems.

Findings

Reduced word diarization error rate (WDER) on TAL and Fisher datasets

Significant improvements in cpWER in the Post-ASR Speaker Tagging Correction challenge

Effective correction of speaker boundary errors in practical speech applications

Abstract

Speech applications dealing with conversations require not only recognizing the spoken words but also determining who spoke when. The task of assigning words to speakers is typically addressed by merging the outputs of two separate systems, namely, an automatic speech recognition (ASR) system and a speaker diarization (SD) system. In practical settings, speaker diarization systems can experience significant degradation in performance due to a variety of factors, including uniform segmentation with a high temporal resolution, inaccurate word timestamps, incorrect clustering and estimation of speaker numbers, as well as background noise. Therefore, it is important to automatically detect errors and make corrections if possible. We used a second-pass speaker tagging correction system based on a non-autoregressive language model to correct mistakes in words placed at the borders of sentences spoken by different speakers. We first show that the employed error correction approach leads to reductions in word diarization error rate (WDER) on two datasets: TAL and test set of Fisher. Additionally, we evaluated our system in the Post-ASR Speaker Tagging Correction challenge and observed significant improvements in cpWER compared to baseline methods.