PoTAcc: A Pipeline for End-to-End Acceleration of Power-of-Two Quantized DNNs

TL;DR

PoTAcc is an open-source pipeline that accelerates and evaluates power-of-two quantized DNNs on resource-constrained edge devices, combining model deployment with custom FPGA accelerators for improved performance and energy efficiency.

Contribution

We introduce PoTAcc, a comprehensive pipeline enabling end-to-end deployment and acceleration of PoT-quantized DNNs on heterogeneous edge hardware, including FPGA-based accelerators.

Findings

Achieves up to 3.6x speedup over CPU-only execution.

Reduces energy consumption by up to 78%.

Supports multiple PoT quantization strategies on FPGA platforms.

Abstract

Power-of-two (PoT) quantization significantly reduces the size of deep neural networks (DNNs) and replaces multiplications with bit-shift operations for inference. Prior work has shown that PoT-quantized DNNs can preserve accuracy for tasks such as image classification; however, their performance on resource-constrained edge devices remains insufficiently understood. While general-purpose edge CPUs and GPUs do not provide optimized backends for bit-shift operations, custom hardware accelerators can better exploit PoT quantization by implementing dedicated shift-based processing elements. However, deploying PoT-quantized models on such accelerators is challenging due to limited support in existing inference frameworks. In addition, the impact of different PoT quantization strategies on hardware design, performance, and energy efficiency during full inference has not been systematically explored. To address these challenges, we propose PoTAcc, an open-source end-to-end pipeline for accelerating and evaluating PoT-quantized DNNs on resource-constrained edge devices. PoTAcc enables seamless preparation and deployment of PoT-quantized models via TensorFlow Lite (TFLite) across heterogeneous platforms, including CPU-only systems and hybrid CPU-FPGA systems with custom accelerators. We design shift-based processing element (shift-PE) accelerators for three PoT quantization methods and implement them on two FPGA platforms. We evaluate accuracy, performance, energy efficiency, and resource utilization across a range of models, including CNNs and Transformer-based architectures. Results show that our CPU-accelerator design achieves up to 3.6x speedup and 78% energy reduction compared to CPU-only execution for PoT-quantized DNNs on PYNQ-Z2 and Kria boards. The code will be publicly released at https://github.com/gicLAB/PoTAcc