Probabilistic Floating-Point Round-Off Analysis via Concentration Inequalities

TL;DR

This paper introduces a scalable probabilistic approach to analyze floating-point round-off errors using concentration inequalities, improving efficiency over existing methods while maintaining accuracy.

Contribution

It presents a novel method applying concentration inequalities to floating-point error analysis, overcoming challenges with absolute values and fractional expressions, and enhances scalability and efficiency.

Findings

Approach is orders of magnitude faster than state-of-the-art methods.

Produces comparable precision in round-off thresholds.

Handles fractional expressions via polynomial transformation.

Abstract

Floating-point round-off errors are ubiquitous in numerically intensive programs arising in fields such as scientific computing and optimization. As floating-point errors potentially lead to unexpected and catastrophic program failures, one must derive guaranteed round-off thresholds to ensure the correctness of these programs. However, deterministic round-off thresholds tend to be too conservative to be usable in practice, since they often involve large round-off errors that occur with small probability. Probabilistic thresholds relax deterministic ones by specifying that the probability of the round-off error exceeding a threshold is below a given confidence. In this work, we propose a novel approach to probabilistic round-off analysis, by applying concentration inequalities over the Taylor expansion from FPTaylor (TOPLAS 2018). A major obstacle in applying concentration inequalities is that the Taylor expansion involves absolute value operators that make the calculation of the expected values of the first order partial differential terms difficult. Our first step to overcome this obstacle is a sound over-approximation that removes the absolute value operators in polynomial expressions. Then, we show how to handle fractional expressions by a transformation into polynomial case. Finally, we show how to improve our approach with range partitioning. Our approach is scalable since the key computational part is the calculation of expected values of polynomial expressions with independent variables, for which the linear and independence properties of expectation boost the computation. Experimental results show that our approach is orders of magnitude more time efficient, while producing thresholds with comparable precision against the state of the art.