The RoyalFlush System of Speech Recognition for M2MeT Challenge

TL;DR

This paper presents the RoyalFlush system for multi-speaker speech recognition in the M2MeT challenge, utilizing advanced front-end processing, extensive data augmentation, and model fusion to significantly improve accuracy.

Contribution

The paper introduces a multi-speaker ASR system that combines WPE, beamforming, data augmentation, and model fusion techniques, achieving state-of-the-art results in the M2MeT challenge.

Findings

12.22% CER reduction on validation set

12.11% CER reduction on test set

Effective combination of front-end processing and model fusion

Abstract

This paper describes our RoyalFlush system for the track of multi-speaker automatic speech recognition (ASR) in the M2MeT challenge. We adopted the serialized output training (SOT) based multi-speakers ASR system with large-scale simulation data. Firstly, we investigated a set of front-end methods, including multi-channel weighted predicted error (WPE), beamforming, speech separation, speech enhancement and so on, to process training, validation and test sets. But we only selected WPE and beamforming as our frontend methods according to their experimental results. Secondly, we made great efforts in the data augmentation for multi-speaker ASR, mainly including adding noise and reverberation, overlapped speech simulation, multi-channel speech simulation, speed perturbation, front-end processing, and so on, which brought us a great performance improvement. Finally, in order to make full use of the performance complementary of different model architecture, we trained the standard conformer based joint CTC/Attention (Conformer) and U2++ ASR model with a bidirectional attention decoder, a modification of Conformer, to fuse their results. Comparing with the official baseline system, our system got a 12.22% absolute Character Error Rate (CER) reduction on the validation set and 12.11% on the test set.