Low-bit Shift Network for End-to-End Spoken Language Understanding

TL;DR

This paper introduces a low-bit shift neural network using power-of-two quantization for end-to-end spoken language understanding, achieving high accuracy with reduced computational complexity suitable for edge devices.

Contribution

It proposes a novel low-bit power-of-two quantization method for shift neural networks, enabling efficient SLU with performance comparable to full-precision models.

Findings

Achieved 98.76% accuracy on test set.

Reduced computational complexity by removing multiplications.

Performed comparably to state-of-the-art solutions.

Abstract

Deep neural networks (DNN) have achieved impressive success in multiple domains. Over the years, the accuracy of these models has increased with the proliferation of deeper and more complex architectures. Thus, state-of-the-art solutions are often computationally expensive, which makes them unfit to be deployed on edge computing platforms. In order to mitigate the high computation, memory, and power requirements of inferring convolutional neural networks (CNNs), we propose the use of power-of-two quantization, which quantizes continuous parameters into low-bit power-of-two values. This reduces computational complexity by removing expensive multiplication operations and with the use of low-bit weights. ResNet is adopted as the building block of our solution and the proposed model is evaluated on a spoken language understanding (SLU) task. Experimental results show improved performance for shift neural network architectures, with our low-bit quantization achieving 98.76 \% on the test set which is comparable performance to its full-precision counterpart and state-of-the-art solutions.