Customizing Number Representation and Precision

TL;DR

This paper compares fixed-point and floating-point number representations in terms of cost, performance, energy, and accuracy, highlighting that the optimal choice depends on specific application needs, especially in AI and FPGA contexts.

Contribution

It provides a detailed comparison of FxP and FlP representations, analyzing their trade-offs and demonstrating scenarios where low-precision floating-point is advantageous.

Findings

Low-precision floating-point can outperform fixed-point in energy-constrained applications.

The choice between FxP and FlP depends heavily on the specific application requirements.

Both representations have distinct advantages and drawbacks in terms of cost, performance, and accuracy.

Abstract

There is a growing interest in the use of reduced-precision arithmetic, exacerbated by the recent interest in artificial intelligence, especially with deep learning. Most architectures already provide reduced-precision capabilities (e.g., 8-bit integer, 16-bit floating point). In the context of FPGAs, any number format and bit-width can even be considered.In computer arithmetic, the representation of real numbers is a major issue. Fixed-point (FxP) and floating-point (FlP) are the main options to represent reals, both with their advantages and drawbacks. This chapter presents both FxP and FlP number representations, and draws a fair a comparison between their cost, performance and energy, as well as their impact on accuracy during computations.It is shown that the choice between FxP and FlP is not obvious and strongly depends on the application considered. In some cases, low-precision floating-point arithmetic can be the most effective and provides some benefits over the classical fixed-point choice for energy-constrained applications.