TL;DR
This paper develops an integer-only quantized Transformer hardware accelerator for embedded FPGA-based time-series forecasting in AIoT, achieving high efficiency with minimal accuracy loss.
Contribution
It introduces a complete FPGA implementation of 4-bit and 6-bit quantized Transformers optimized for AIoT, demonstrating significant improvements in speed and energy efficiency.
Findings
4-bit Transformer increases test loss by only 0.63%
Operates up to 132.33x faster than 8-bit models
Consumes 48.19x less energy than 8-bit models
Abstract
This paper presents the design of a hardware accelerator for Transformers, optimized for on-device time-series forecasting in AIoT systems. It integrates integer-only quantization and Quantization-Aware Training with optimized hardware designs to realize 6-bit and 4-bit quantized Transformer models, which achieved precision comparable to 8-bit quantized models from related research. Utilizing a complete implementation on an embedded FPGA (Xilinx Spartan-7 XC7S15), we examine the feasibility of deploying Transformer models on embedded IoT devices. This includes a thorough analysis of achievable precision, resource utilization, timing, power, and energy consumption for on-device inference. Our results indicate that while sufficient performance can be attained, the optimization process is not trivial. For instance, reducing the quantization bitwidth does not consistently result in…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
