ASRPU: A Programmable Accelerator for Low-Power Automatic Speech Recognition

TL;DR

This paper introduces ASRPU, a programmable hardware accelerator designed for low-power, real-time automatic speech recognition on edge devices, balancing flexibility and efficiency.

Contribution

The paper presents ASRPU, a novel programmable accelerator with general-purpose cores that simplifies on-edge ASR deployment while maintaining adaptability to evolving models.

Findings

Achieves real-time decoding on edge devices

Operates with very low power consumption

Supports flexible programming for different ASR models

Abstract

The outstanding accuracy achieved by modern Automatic Speech Recognition (ASR) systems is enabling them to quickly become a mainstream technology. ASR is essential for many applications, such as speech-based assistants, dictation systems and real-time language translation. However, highly accurate ASR systems are computationally expensive, requiring on the order of billions of arithmetic operations to decode each second of audio, which conflicts with a growing interest in deploying ASR on edge devices. On these devices, hardware acceleration is key for achieving acceptable performance. However, ASR is a rich and fast-changing field, and thus, any overly specialized hardware accelerator may quickly become obsolete. In this paper, we tackle those challenges by proposing ASRPU, a programmable accelerator for on-edge ASR. ASRPU contains a pool of general-purpose cores that execute small pieces of parallel code. Each of these programs computes one part of the overall decoder (e.g. a layer in a neural network). The accelerator automates some carefully chosen parts of the decoder to simplify the programming without sacrificing generality. We provide an analysis of a modern ASR system implemented on ASRPU and show that this architecture can achieve real-time decoding with a very low power budget.