Bias- and Variance-Aware Probabilistic Rounding Error Analysis for Floating-Point Arithmetic

TL;DR

This paper develops a probabilistic error analysis framework for floating-point arithmetic that accounts for bias and variance, providing sharper bounds especially in low-precision computations.

Contribution

It introduces a variance-informed probabilistic backward error bound that explicitly incorporates bias and variance, extending previous zero-mean models.

Findings

Bias-aware models show different error growth behaviors.

The framework provides tighter bounds in low-precision regimes.

Experiments confirm the practical usefulness of the bias-aware analysis.

Abstract

Probabilistic rounding error analysis can yield much sharper bounds than classical worst-case theory, but existing results typically rely on zero-mean rounding errors and often leave the confidence parameter implicit. This work revisits probabilistic rounding error analysis in a moment-aware setting. We first derive a confidence-calibrated reformulation of the Higham and Mary [16] bound that makes its confidence parameter explicit. We then introduce a variance-informed probabilistic backward error bound based on the first two moments of $\log(1+\delta)$, where $\delta$ is the relative rounding error. This allows the analysis to accommodate biased rounding error models rather than relying on a zero-mean assumption. To illustrate this framework, we study both a uniform model and a log-space $\operatorname{Beta}$ model for rounding errors, the latter of which provides a simple way to represent bias. This perspective shows that the growth of probabilistic rounding error bounds is not universal: near-zero-mean regimes recover $\sqrt{n}$-like behavior, while biased models can exhibit faster accumulation. $\texttt{CUDA}$ experiments in single and half precision on dot products, sparse matrix-vector products, and a stochastic boundary-value problem show that the proposed framework is especially useful in low-precision regimes where deterministic bounds are overly conservative and where bias-aware modeling better matches observed error growth.