HOBFLOPS CNNs: Hardware Optimized Bitslice-Parallel Floating-Point Operations for Convolutional Neural Networks

TL;DR

HOBFLOPS introduces hardware-optimized bitslice-parallel floating-point operations enabling efficient, customizable low-precision CNN inference on general-purpose processors, significantly outperforming existing softFP16 implementations.

Contribution

The paper presents a novel method to generate hardware-optimized, custom-precision bitslice-parallel floating-point routines for CNNs, bridging the gap between FPGA/ASIC accelerators and general-purpose processors.

Findings

HOBFLOPS16 outperforms SoftFP16 by 8x on Intel AVX512.

HOBFLOPS9 achieves 6x the performance of HOBFLOPS16 on Arm Neon.

HOBFLOPS enables flexible, high-performance low-precision CNN inference on standard CPUs.

Abstract

Convolutional neural networks (CNNs) are typically trained using 16- or 32-bit floating-point (FP) and researchers show that low-precision floating-point (FP) can be highly effective for inference. Low-precision FP can be implemented in field programmable gate array (FPGA) and application-specific integrated circuit (ASIC) accelerators, but existing processors do not generally support custom precision FP. We propose hardware optimized bitslice-parallel floating-point operators (HOBFLOPS), a method of generating efficient custom-precision emulated bitslice-parallel software FP arithmetic. We generate custom-precision FP routines optimized using a hardware synthesis design flow to create circuits. We provide standard cell libraries matching the bitwise operations on the target microprocessor architecture, and a code-generator to translate the hardware circuits to bitslice software equivalents. We exploit bitslice parallelism to create a very wide (32-512 element) vectorized convolutional neural network (CNN) convolution. Hardware optimized bitslice-parallel floating-point operators (HOBFLOPS) multiply-accumulate (MAC) performance in CNN convolution on Arm and Intel processors are compared to Berkeley's SoftFP16 equivalent MAC. HOBFLOPS16 outperforms SoftFP16 by 8x on Intel AVX512. HOBFLOPS offers arbitrary-precision FP with custom range and precision e.g., HOBFLOPS9 performs at 6x the performance of HOBFLOPS16 on Arm Neon. HOBFLOPS allows researchers to prototype different levels of custom FP precision in the arithmetic of software CNN accelerators. Furthermore, HOBFLOPS fast custom-precision FP CNNs may be valuable in cases where memory bandwidth is limited.