Exploiting Beam Search Confidence for Energy-Efficient Speech Recognition

TL;DR

This paper introduces a method to enhance energy efficiency and reduce computation in DNN-based speech recognition systems by utilizing beam search confidence, achieving significant energy and time savings with minimal accuracy loss.

Contribution

It presents a novel technique that leverages run-time beam search confidence to optimize acoustic model evaluation for low-power edge devices.

Findings

Reduced energy consumption by 25.6%

Decreased execution time by 25.9%

Maintained negligible accuracy loss

Abstract

With computers getting more and more powerful and integrated in our daily lives, the focus is increasingly shifting towards more human-friendly interfaces, making Automatic Speech Recognition (ASR) a central player as the ideal means of interaction with machines. Consequently, interest in speech technology has grown in the last few years, with more systems being proposed and higher accuracy levels being achieved, even surpassing \textit{Human Accuracy}. While ASR systems become increasingly powerful, the computational complexity also increases, and the hardware support have to keep pace. In this paper, we propose a technique to improve the energy-efficiency and performance of ASR systems, focusing on low-power hardware for edge devices. We focus on optimizing the DNN-based Acoustic Model evaluation, as we have observed it to be the main bottleneck in state-of-the-art ASR systems, by leveraging run-time information from the Beam Search. By doing so, we reduce energy and execution time of the acoustic model evaluation by 25.6% and 25.9%, respectively, with negligible accuracy loss.