Limited-Precision Stochastic Rounding

TL;DR

This paper reviews recent advances in limited-precision stochastic rounding, a probabilistic method for rounding in low-precision computations, highlighting its benefits, recent research developments, and potential for hardware implementation.

Contribution

It provides an update on stochastic rounding research since 2022, focusing on a new limited-precision variant and discussing its industrial and numerical analysis applications.

Findings

SR error grows as √n in summation, unlike RN which grows as n

SR alleviates stagnation by including small summands

Recent research has expanded understanding and implementation of SR

Abstract

Stochastic rounding (SR) is a probabilistic method used to round numbers to floating-point and fixed-point representations. In length $n$ summation, the worst-case error of SR grows as $\sqrt{n}$ with high probability, unlike for standard modes, like round-to-nearest (RN), which grows as $n$. For this reason, the former is increasingly employed in large-scale, low-precision computations as an RN alternative. Additionally, SR alleviates stagnation, whereby relatively small summands are completely rounded off and do not contribute to the sum. We provide an update to [Croci et al., Roy. Soc. Open Sci. 9.3 (2022), pp. 1-25], a survey which discusses the development and use of SR between 1949 and 2022, citing over 100 references. Since then, there has been a surge of new research, and this update covers almost four years of further progress in applying, analysing, and implementing SR. Our main focus is limited-precision stochastic rounding, a new variant that fixes the precision of the random numbers used. We provide insights into industrial and numerical analysis activities surrounding SR, highlighting the next possible steps in making this rounding mode more widely available in hardware.