The RoyalFlush Automatic Speech Diarization and Recognition System for In-Car Multi-Channel Automatic Speech Recognition Challenge

TL;DR

This paper introduces an end-to-end system for multi-channel in-car speech recognition and diarization, significantly reducing errors and improving accuracy in complex multi-speaker scenarios.

Contribution

We develop novel end-to-end diarization models that greatly lower diarization errors and integrate self-supervised learning for improved speech recognition in challenging environments.

Findings

Diarization error rate reduced by 49.58%

Character error rate achieved is 16.93% on evaluation set

cpCER of 25.88% on track 2 evaluation set

Abstract

This paper presents our system submission for the In-Car Multi-Channel Automatic Speech Recognition (ICMC-ASR) Challenge, which focuses on speaker diarization and speech recognition in complex multi-speaker scenarios. To address these challenges, we develop end-to-end speaker diarization models that notably decrease the diarization error rate (DER) by 49.58\% compared to the official baseline on the development set. For speech recognition, we utilize self-supervised learning representations to train end-to-end ASR models. By integrating these models, we achieve a character error rate (CER) of 16.93\% on the track 1 evaluation set, and a concatenated minimum permutation character error rate (cpCER) of 25.88\% on the track 2 evaluation set.