Probabilistic error analysis of limited-precision stochastic rounding

TL;DR

This paper analyzes how limited-precision stochastic rounding, constrained by finite random bits, introduces bias in floating-point computations and develops a probabilistic error model that reflects practical hardware implementations.

Contribution

It introduces limited-precision stochastic rounding, models its bias depending on random bits, and provides a probabilistic error analysis aligned with real hardware constraints.

Findings

Limited-precision SR is biased, with bias depending on random bits used.

As the number of random bits increases, bias diminishes and converges to ideal SR.

Numerical examples confirm the theoretical bias and error bounds.

Abstract

Classical probabilistic rounding error analysis is particularly well suited to stochastic rounding (SR), and it yields strong results when dealing with floating-point algorithms that rely heavily on summation. For many numerical linear algebra algorithms, one can prove probabilistic error bounds that grow as O(nu), where n is the problem size and u is the unit roundoff. These probabilistic bounds are asymptotically tighter than the worst-case ones, which grow as O(nu). For certain classes of algorithms, SR has been shown to be unbiased. However, all these results were derived under the assumption that SR is implemented exactly, which typically requires a number of random bits that is too large to be suitable for practical implementations. We investigate the effect of the number of random bits on the probabilistic rounding error analysis of SR. To this end, we introduce a new rounding mode, limited-precision SR. By taking into account the number r of random bits used, this new rounding mode matches hardware implementations accurately, unlike the ideal SR operator generally used in the literature. We show that this new rounding mode is biased and that the bias is a function of r. As r approaches infinity, however, the bias disappears, and limited-precision SR converges to the ideal, unbiased SR operator. We develop a novel model for probabilistic error analysis of algorithms employing SR. Several numerical examples corroborate our theoretical findings.