Hardware Acceleration of Fully Quantized BERT for Efficient Natural Language Processing

TL;DR

This paper presents a fully quantized BERT model optimized for FPGA hardware, achieving significant compression and energy-efficient performance improvements for NLP tasks at the edge.

Contribution

The paper introduces FQ-BERT, a fully quantized version of BERT, and designs a specialized FPGA accelerator to enhance efficiency and deployment at the edge.

Findings

FQ-BERT achieves 7.94x weight compression with negligible performance loss.

The FPGA accelerator delivers 3.18 fps/W, outperforming CPU and GPU in energy efficiency.

Significant reduction in computational complexity and memory footprint for BERT.

Abstract

BERT is the most recent Transformer-based model that achieves state-of-the-art performance in various NLP tasks. In this paper, we investigate the hardware acceleration of BERT on FPGA for edge computing. To tackle the issue of huge computational complexity and memory footprint, we propose to fully quantize the BERT (FQ-BERT), including weights, activations, softmax, layer normalization, and all the intermediate results. Experiments demonstrate that the FQ-BERT can achieve 7.94x compression for weights with negligible performance loss. We then propose an accelerator tailored for the FQ-BERT and evaluate on Xilinx ZCU102 and ZCU111 FPGA. It can achieve a performance-per-watt of 3.18 fps/W, which is 28.91x and 12.72x over Intel(R) Core(TM) i7-8700 CPU and NVIDIA K80 GPU, respectively.