Temporal Feedback Convolutional Recurrent Neural Networks for Speech Command Recognition

TL;DR

This paper introduces a novel neural network architecture for speech command recognition that incorporates temporal feedback control inspired by human auditory mechanisms, leading to improved performance over existing CNN-based models.

Contribution

It extends SENets with a recurrent module for temporal feedback, enhancing feature modulation in speech recognition tasks.

Findings

The proposed model slightly outperforms SENets and other CNN models.

Temporal feedback improves feature scaling and recognition accuracy.

Failure analysis reveals insights into the model's performance improvements.

Abstract

End-to-end learning models using raw waveforms as input have shown superior performances in many audio recognition tasks. However, most model architectures are based on convolutional neural networks (CNN) which were mainly developed for visual recognition tasks. In this paper, we propose an extension of squeeze-and-excitation networks (SENets) which adds temporal feedback control from the top-layer features to channel-wise feature activations in lower layers using a recurrent module. This is analogous to the adaptive gain control mechanism of outer hair-cell in the human auditory system. We apply the proposed model to speech command recognition and show that it slightly outperforms the SENets and other CNN-based models. We also investigate the details of the performance improvement by conducting failure analysis and visualizing the channel-wise feature scaling induced by the temporal feedback.