Long Short-Term Memory based Convolutional Recurrent Neural Networks for Large Vocabulary Speech Recognition

TL;DR

This paper introduces a novel convolutional recurrent neural network (CRNN) architecture combining CNNs and LSTM RNNs, achieving superior performance in large vocabulary speech recognition tasks.

Contribution

The paper proposes a new CRNN architecture that integrates CNNs and LSTM RNNs for improved speech recognition accuracy, demonstrating its effectiveness through extensive experiments.

Findings

LSTM CRNNs outperform traditional FFNNs and standalone LSTM RNNs.

The proposed architecture exceeds state-of-the-art speech recognition performance.

Experimental results validate the effectiveness of combining CNNs with LSTM RNNs.

Abstract

Long short-term memory (LSTM) recurrent neural networks (RNNs) have been shown to give state-of-the-art performance on many speech recognition tasks, as they are able to provide the learned dynamically changing contextual window of all sequence history. On the other hand, the convolutional neural networks (CNNs) have brought significant improvements to deep feed-forward neural networks (FFNNs), as they are able to better reduce spectral variation in the input signal. In this paper, a network architecture called as convolutional recurrent neural network (CRNN) is proposed by combining the CNN and LSTM RNN. In the proposed CRNNs, each speech frame, without adjacent context frames, is organized as a number of local feature patches along the frequency axis, and then a LSTM network is performed on each feature patch along the time axis. We train and compare FFNNs, LSTM RNNs and the proposed LSTM CRNNs at various number of configurations. Experimental results show that the LSTM CRNNs can exceed state-of-the-art speech recognition performance.