Quark: An Integer RISC-V Vector Processor for Sub-Byte Quantized DNN Inference

TL;DR

Quark is a specialized integer RISC-V vector processor optimized for efficient sub-byte quantized DNN inference, achieving significant size and power improvements over existing designs.

Contribution

It introduces a novel extension of the Ara processor with sub-byte vector instructions and removes floating-point units to optimize for quantized neural network inference.

Findings

Quark can run 1-bit and 2-bit quantized models effectively.

It accelerates Conv2d computations for various input and kernel sizes.

Quark is more power-efficient and smaller than Ara.

Abstract

In this paper, we present Quark, an integer RISC-V vector processor specifically tailored for sub-byte DNN inference. Quark is implemented in GlobalFoundries' 22FDX FD-SOI technology. It is designed on top of Ara, an open-source 64-bit RISC-V vector processor. To accommodate sub-byte DNN inference, Quark extends Ara by adding specialized vector instructions to perform sub-byte quantized operations. We also remove the floating-point unit from Quarks' lanes and use the CVA6 RISC-V scalar core for the re-scaling operations that are required in quantized neural network inference. This makes each lane of Quark 2 times smaller and 1.9 times more power efficient compared to the ones of Ara. In this paper we show that Quark can run quantized models at sub-byte precision. Notably we show that for 1-bit and 2-bit quantized models, Quark can accelerate computation of Conv2d over various ranges of inputs and kernel sizes.