MFCCA:Multi-Frame Cross-Channel attention for multi-speaker ASR in Multi-party meeting scenario

TL;DR

This paper introduces a multi-frame cross-channel attention mechanism for multi-speaker ASR in multi-party meetings, leveraging adjacent frame information to improve recognition accuracy over existing methods.

Contribution

It proposes a novel multi-frame cross-channel attention model with a convolutional fusion mechanism and channel masking, achieving state-of-the-art results on AliMeeting.

Findings

Outperforms single-channel models with 31.7% and 37.0% CER reduction.

Achieves new SOTA performance on AliMeeting corpus.

Effectively handles channel number mismatch during training and inference.

Abstract

Recently cross-channel attention, which better leverages multi-channel signals from microphone array, has shown promising results in the multi-party meeting scenario. Cross-channel attention focuses on either learning global correlations between sequences of different channels or exploiting fine-grained channel-wise information effectively at each time step. Considering the delay of microphone array receiving sound, we propose a multi-frame cross-channel attention, which models cross-channel information between adjacent frames to exploit the complementarity of both frame-wise and channel-wise knowledge. Besides, we also propose a multi-layer convolutional mechanism to fuse the multi-channel output and a channel masking strategy to combat the channel number mismatch problem between training and inference. Experiments on the AliMeeting, a real-world corpus, reveal that our proposed model outperforms single-channel model by 31.7\% and 37.0\% CER reduction on Eval and Test sets. Moreover, with comparable model parameters and training data, our proposed model achieves a new SOTA performance on the AliMeeting corpus, as compared with the top ranking systems in the ICASSP2022 M2MeT challenge, a recently held multi-channel multi-speaker ASR challenge.