Quaternion Neural Networks for Multi-channel Distant Speech Recognition

TL;DR

This paper introduces quaternion neural networks for multi-channel distant speech recognition, effectively modeling inter-channel dependencies and improving recognition accuracy over traditional real-valued models.

Contribution

The paper presents a novel quaternion neural network architecture, specifically a quaternion LSTM, for processing multi-channel audio in distant speech recognition tasks.

Findings

Quaternion LSTM outperforms real-valued LSTM in recognition accuracy.

Quaternion algebra effectively models inter-channel dependencies.

The approach improves robustness in noisy, reverberant environments.

Abstract

Despite the significant progress in automatic speech recognition (ASR), distant ASR remains challenging due to noise and reverberation. A common approach to mitigate this issue consists of equipping the recording devices with multiple microphones that capture the acoustic scene from different perspectives. These multi-channel audio recordings contain specific internal relations between each signal. In this paper, we propose to capture these inter- and intra- structural dependencies with quaternion neural networks, which can jointly process multiple signals as whole quaternion entities. The quaternion algebra replaces the standard dot product with the Hamilton one, thus offering a simple and elegant way to model dependencies between elements. The quaternion layers are then coupled with a recurrent neural network, which can learn long-term dependencies in the time domain. We show that a quaternion long-short term memory neural network (QLSTM), trained on the concatenated multi-channel speech signals, outperforms equivalent real-valued LSTM on two different tasks of multi-channel distant speech recognition.