Deep Learning Inference on Embedded Devices: Fixed-Point vs Posit

TL;DR

This paper explores using the posit number system for deep learning inference on embedded devices, demonstrating improved accuracy and memory efficiency over traditional fixed-point systems without re-training.

Contribution

It introduces the application of the posit number system to CNN weights, showing benefits over fixed-point representation without needing re-training.

Findings

Posit outperforms fixed-point in accuracy.

Posit reduces memory usage.

No re-training required for weights.

Abstract

Performing the inference step of deep learning in resource constrained environments, such as embedded devices, is challenging. Success requires optimization at both software and hardware levels. Low precision arithmetic and specifically low precision fixed-point number systems have become the standard for performing deep learning inference. However, representing non-uniform data and distributed parameters (e.g. weights) by using uniformly distributed fixed-point values is still a major drawback when using this number system. Recently, the posit number system was proposed, which represents numbers in a non-uniform manner. Therefore, in this paper we are motivated to explore using the posit number system to represent the weights of Deep Convolutional Neural Networks. However, we do not apply any quantization techniques and hence the network weights do not require re-training. The results of this exploration show that using the posit number system outperformed the fixed point number system in terms of accuracy and memory utilization.