Balancing Inexactness in Mixed Precision Matrix Computations

TL;DR

This paper explores how mixed precision arithmetic can be safely used in scientific computations by balancing inexactness sources to enhance performance without sacrificing accuracy, supported by recent numerical linear algebra examples.

Contribution

It introduces a framework for analyzing and balancing inexactness from various sources to optimize mixed precision computations in scientific applications.

Findings

Mixed precision can be effectively balanced with other inexactness sources.

Examples show potential performance gains in matrix computations.

Analysis guides precision choices to maintain accuracy.

Abstract

Support for arithmetic in multiple precisions and number formats is becoming increasingly common in emerging high-performance architectures. From a computational scientist's perspective, our goal is to determine how and where we can safely exploit mixed precision computation in our codes to improve performance. One case where the use of low precision is natural, common in computational science, is when there are already other significant sources of ``inexactness'' present, e.g., discretization error, measurement error, or algorithmic approximation error. In such instances, analyzing the interaction of these different sources of inexactness can give insight into how the precisions of various computations should be chosen in order to ``balance'' the errors, potentially improving performance without a noticeable decrease in accuracy. We present a few recent examples of this approach which demonstrate the potential for the use of mixed precision in numerical linear algebra and matrix computations.