HaShiFlex: A High-Throughput Hardened Shifter DNN Accelerator with Fine-Tuning Flexibility

TL;DR

HaShiFlex is a high-throughput, energy-efficient DNN accelerator that embeds most layers in hardware, uses power-of-two quantization for simplified computation, and maintains fine-tuning flexibility, significantly outperforming GPUs in inference speed.

Contribution

The paper presents a novel hardware architecture that combines embedded layer processing, power-of-two quantization, and a reprogrammable final layer for high throughput and fine-tuning capability.

Findings

Achieves 20x inference throughput increase over GPUs with fine-tuning.

Processes 1.21 million images/sec with full fine-tuning.

Reaches 4 million images/sec without post-deployment fine-tuning.

Abstract

We introduce a high-throughput neural network accelerator that embeds most network layers directly in hardware, minimizing data transfer and memory usage while preserving a degree of flexibility via a small neural processing unit for the final classification layer. By leveraging power-of-two (Po2) quantization for weights, we replace multiplications with simple rewiring, effectively reducing each convolution to a series of additions. This streamlined approach offers high-throughput, energy-efficient processing, making it highly suitable for applications where model parameters remain stable, such as continuous sensing tasks at the edge or large-scale data center deployments. Furthermore, by including a strategically chosen reprogrammable final layer, our design achieves high throughput without sacrificing fine-tuning capabilities. We implement this accelerator in a 7nm ASIC flow using MobileNetV2 as a baseline and report throughput, area, accuracy, and sensitivity to quantization and pruning - demonstrating both the advantages and potential trade-offs of the proposed architecture. We find that for MobileNetV2, we can improve inference throughput by 20x over fully programmable GPUs, processing 1.21 million images per second through a full forward pass while retaining fine-tuning flexibility. If absolutely no post-deployment fine tuning is required, this advantage increases to 67x at 4 million images per second.