Probabilistic Error Analysis for Inner Products

TL;DR

This paper introduces probabilistic models to bound the forward error in computed inner products, providing explicit, non-asymptotic bounds that often outperform traditional deterministic bounds, especially for small dimensions.

Contribution

It develops novel probabilistic bounds for inner product errors using Azuma's inequality and Martingales, with minimal assumptions on perturbations and roundoffs.

Findings

Probabilistic bounds are often several orders of magnitude tighter than deterministic bounds.

Roundoff error bounds scale with  rather than n, confirming Wilkinson's intuition.

Numerical experiments demonstrate the effectiveness of the probabilistic bounds across various scenarios.

Abstract

Probabilistic models are proposed for bounding the forward error in the numerically computed inner product (dot product, scalar product) between of two real $n$-vectors. We derive probabilistic perturbation bounds, as well as probabilistic roundoff error bounds for the sequential accumulation of the inner product. These bounds are non-asymptotic, explicit, and make minimal assumptions on perturbations and roundoffs. The perturbations are represented as independent, bounded, zero-mean random variables, and the probabilistic perturbation bound is based on Azuma's inequality. The roundoffs are also represented as bounded, zero-mean random variables. The first probabilistic bound assumes that the roundoffs are independent, while the second one does not. For the latter, we construct a Martingale that mirrors the sequential order of computations. Numerical experiments confirm that our bounds are more informative, often by several orders of magnitude, than traditional deterministic bounds -- even for small vector dimensions~$n$ and very stringent success probabilities. In particular the probabilistic roundoff error bounds are functions of $\sqrt{n}$ rather than~$n$, thus giving a quantitative confirmation of Wilkinson's intuition. The paper concludes with a critical assessment of the probabilistic approach.