End-to-end Multichannel Speaker-Attributed ASR: Speaker Guided Decoder and Input Feature Analysis

TL;DR

This paper introduces a novel multichannel speaker-attributed ASR system that integrates ASR and speaker identification, demonstrating significant WER improvements on simulated and real-world datasets.

Contribution

It presents the first efficient integration of ASR and speaker ID modules in a multichannel framework with detailed input feature analysis.

Findings

Up to 16% relative WER reduction on simulated data

Effective performance on real-world AMI meeting data

Impact of multichannel features on ASR accuracy

Abstract

We present an end-to-end multichannel speaker-attributed automatic speech recognition (MC-SA-ASR) system that combines a Conformer-based encoder with multi-frame crosschannel attention and a speaker-attributed Transformer-based decoder. To the best of our knowledge, this is the first model that efficiently integrates ASR and speaker identification modules in a multichannel setting. On simulated mixtures of LibriSpeech data, our system reduces the word error rate (WER) by up to 12% and 16% relative compared to previously proposed single-channel and multichannel approaches, respectively. Furthermore, we investigate the impact of different input features, including multichannel magnitude and phase information, on the ASR performance. Finally, our experiments on the AMI corpus confirm the effectiveness of our system for real-world multichannel meeting transcription.