A Probabilistic Approach to Floating-Point Arithmetic

TL;DR

This paper introduces a probabilistic model for floating-point rounding errors that estimates the likelihood of errors within specific bounds, enabling more realistic error analysis compared to traditional worst-case bounds.

Contribution

It develops a compositional probabilistic framework for analyzing floating-point errors in programs, including exact and approximate error distribution computations.

Findings

Exact error distribution for low precision arithmetic

Approximate error distribution for high precision arithmetic

Probabilistic range analysis applied to benchmarks

Abstract

Finite-precision floating point arithmetic unavoidably introduces rounding errors which are traditionally bounded using a worst-case analysis. However, worst-case analysis might be overly conservative because worst-case errors can be extremely rare events in practice. Here we develop a probabilistic model of rounding errors with which it becomes possible to estimate the likelihood that the rounding error of an algorithm lies within a given interval. Given an input distribution, we show how to compute the distribution of rounding errors. We do this exactly for low precision arithmetic, for high precision arithmetic we derive a simple approximation. The model is then entirely compositional: given a numerical program written in a simple imperative programming language we can recursively compute the distribution of rounding errors at each step of the computation and propagate it through each program instruction. This is done by applying a formalism originally developed by Kozen to formalize the semantics of probabilistic programs. We then discuss an implementation of the model and use it to perform probabilistic range analyses on some benchmarks.