Towards Structured Deep Neural Network for Automatic Speech Recognition

TL;DR

This paper introduces a Structured Deep Neural Network (Structured DNN) for automatic speech recognition, enabling holistic learning of utterances by considering the entire structure, and demonstrates its superiority over structured SVM on TIMIT.

Contribution

The paper proposes a novel Structured DNN that incorporates deep learning for structured output prediction in speech recognition, surpassing structured SVM performance.

Findings

Structured DNN outperforms structured SVM on TIMIT.

Deep learning enhances structured prediction in speech recognition.

Holistic utterance modeling improves recognition accuracy.

Abstract

In this paper we propose the Structured Deep Neural Network (Structured DNN) as a structured and deep learning algorithm, learning to find the best structured object (such as a label sequence) given a structured input (such as a vector sequence) by globally considering the mapping relationships between the structure rather than item by item. When automatic speech recognition is viewed as a special case of such a structured learning problem, where we have the acoustic vector sequence as the input and the phoneme label sequence as the output, it becomes possible to comprehensively learned utterance by utterance as a whole, rather than frame by frame. Structured Support Vector Machine (structured SVM) was proposed to perform ASR with structured learning previously, but limited by the linear nature of SVM. Here we propose structured DNN to use nonlinear transformations in multi-layers as a structured and deep learning algorithm. It was shown to beat structured SVM in preliminary experiments on TIMIT.