Error Analysis of Sum-Product Algorithms under Stochastic Rounding

TL;DR

This paper introduces a general probabilistic error analysis method for algorithms under stochastic rounding, providing tighter bounds and applying it to various algorithms including polynomial multiplication.

Contribution

It develops a universal martingale construction technique for multi-linear computations and extends stochastic rounding error bounds to new algorithms like Karatsuba multiplication.

Findings

Error bounds are probabilistic and tighter than worst-case bounds.

The method applies to algorithms with reused intermediate computations.

It successfully analyzes Karatsuba polynomial multiplication.

Abstract

The quality of numerical computations can be measured through their forward error, for which finding good error bounds is challenging in general. For several algorithms and using stochastic rounding (SR), probabilistic analysis has been shown to be an effective alternative for obtaining tight error bounds. This analysis considers the distribution of errors and evaluates the algorithm's performance on average. Using martingales and the Azuma-Hoeffding inequality, it provides error bounds that are valid with a certain probability and in O($\sqrt$nu) instead of deterministic worst-case bounds in O(nu), where n is the number of operations and u is the unit roundoff. In this paper, we present a general method that automatically constructs a martingale for any computation scheme with multi-linear errors based on additions, subtractions, and multiplications. We apply this generalization to algorithms previously studied with SR, such as pairwise summation and the Horner algorithm, and prove equivalent results. We also analyze a previously unstudied algorithm, Karatsuba polynomial multiplication, which illustrates that the method can handle reused intermediate computations.