Dekker's floating point number system and compensated summation algorithms

TL;DR

This paper analyzes Dekker's floating point number system and evaluates compensated summation algorithms to improve numerical accuracy in reduced precision computing, providing theoretical insights and empirical validation.

Contribution

It offers a detailed analysis of Dekker's number system and introduces improved compensated summation techniques with error behavior descriptions and practical applications.

Findings

Dekker's system facilitates analysis of error propagation in summation.

Compensated summation techniques significantly enhance accuracy.

Empirical results confirm theoretical error bounds.

Abstract

The recent hardware trend towards reduced precision computing has reignited the interest in numerical techniques that can be used to enhance the accuracy of floating point operations beyond what is natively supported for basic arithmetic operations on the hardware. In this work, we study the behavior of various compensated summation techniques, which can be used to enhance the accuracy for the summation operation, particularly in situations when the addends are not known a priori. Complete descriptions of the error behavior are provided for these techniques. In particular, the relationship between the intermediate results at two consecutive summing steps is provided, which is used to identify the operation that limits accuracy and guide the design of more nuanced techniques. The analysis relies on the work of Dekker [Numerische Mathematik, 1971], which uses a special floating point number system that does not require uniqueness in the number representation. Despite this seemingly strange attribute, Dekker's system is very convenient for the analysis here and allows general statements to be expressed succinctly compared to a number system that requires uniqueness. To prepare the foundation for discussion, we start by giving a thorough exhibition of Dekker's number system and supply the details that were omitted in Dekker [Numerische Mathematik, 1971]. Numerical examples are designed to explain the inner workings of these compensated summation techniques, illustrate their efficacy, and empirically verify the analytical results derived. Discussions are also given on application scenarios where these techniques can be beneficial.