A Mixed-Precision RISC-V Processor for Extreme-Edge DNN Inference

TL;DR

This paper introduces MPIC, a novel RISC-V processor extension supporting mixed-precision quantized neural network inference on microcontrollers, significantly improving performance and energy efficiency without increasing ISA complexity.

Contribution

The work presents a status-based SIMD instruction set extension for RISC-V that enables dynamic mixed-precision QNN inference support without extra opcodes or decode complexity.

Findings

MPIC improves performance and energy efficiency by up to 4.9x compared to software implementations.

It outperforms Cortex-M4 and M7 microcontrollers by up to 11.7x in performance.

Supports 16-, 8-, 4-, and 2-bit precision for mixed-precision QNNs.

Abstract

Low bit-width Quantized Neural Networks (QNNs) enable deployment of complex machine learning models on constrained devices such as microcontrollers (MCUs) by reducing their memory footprint. Fine-grained asymmetric quantization (i.e., different bit-widths assigned to weights and activations on a tensor-by-tensor basis) is a particularly interesting scheme to maximize accuracy under a tight memory constraint. However, the lack of sub-byte instruction set architecture (ISA) support in SoA microprocessors makes it hard to fully exploit this extreme quantization paradigm in embedded MCUs. Support for sub-byte and asymmetric QNNs would require many precision formats and an exorbitant amount of opcode space. In this work, we attack this problem with status-based SIMD instructions: rather than encoding precision explicitly, each operand's precision is set dynamically in a core status register. We propose a novel RISC-V ISA core MPIC (Mixed Precision Inference Core) based on the open-source RI5CY core. Our approach enables full support for mixed-precision QNN inference with different combinations of operands at 16-, 8-, 4- and 2-bit precision, without adding any extra opcode or increasing the complexity of the decode stage. Our results show that MPIC improves both performance and energy efficiency by a factor of 1.1-4.9x when compared to software-based mixed-precision on RI5CY; with respect to commercially available Cortex-M4 and M7 microcontrollers, it delivers 3.6-11.7x better performance and 41-155x higher efficiency.