Acoustic Landmarks Contain More Information About the Phone String than Other Frames for Automatic Speech Recognition with Deep Neural Network Acoustic Model

TL;DR

This paper demonstrates that acoustic landmark frames are more informative for speech recognition than other frames, and leveraging them can improve accuracy and reduce computational load in DNN-based ASR systems.

Contribution

It provides empirical evidence that acoustic landmarks contain more information about phone strings and introduces landmark-based strategies to enhance ASR performance and efficiency.

Findings

Landmark frames are more informative for ASR.

Re-weighting landmarks reduces phone error rate.

Landmark-based frame dropping maintains accuracy with fewer frames.

Abstract

Most mainstream Automatic Speech Recognition (ASR) systems consider all feature frames equally important. However, acoustic landmark theory is based on a contradictory idea, that some frames are more important than others. Acoustic landmark theory exploits quantal non-linearities in the articulatory-acoustic and acoustic-perceptual relations to define landmark times at which the speech spectrum abruptly changes or reaches an extremum; frames overlapping landmarks have been demonstrated to be sufficient for speech perception. In this work, we conduct experiments on the TIMIT corpus, with both GMM and DNN based ASR systems and find that frames containing landmarks are more informative for ASR than others. We find that altering the level of emphasis on landmarks by re-weighting acoustic likelihood tends to reduce the phone error rate (PER). Furthermore, by leveraging the landmark as a heuristic, one of our hybrid DNN frame dropping strategies maintained a PER within 0.44% of optimal when scoring less than half (45.8% to be precise) of the frames. This hybrid strategy out-performs other non-heuristic-based methods and demonstrate the potential of landmarks for reducing computation.