Effects of round-to-nearest and stochastic rounding in the numerical solution of the heat equation in low precision

TL;DR

This paper compares round-to-nearest and stochastic rounding in solving the heat equation with low-precision arithmetic, showing stochastic rounding's superior error resilience and bounded error growth.

Contribution

It provides a detailed analysis of rounding error accumulation in low-precision PDE solutions, introducing stochastic rounding as a more stable alternative to round-to-nearest.

Findings

Stochastic rounding errors are zero-mean and independent, reducing error accumulation.

Round-to-nearest causes solution stagnation and higher error growth.

Stochastic rounding maintains bounded global errors in multiple dimensions.

Abstract

Motivated by the advent of machine learning, the last few years have seen the return of hardware-supported low-precision computing. Computations with fewer digits are faster and more memory and energy efficient, but can be extremely susceptible to rounding errors. As shown by recent studies into reduced-precision climate simulations, an application that can largely benefit from the advantages of low-precision computing is the numerical solution of partial differential equations (PDEs). However, a careful implementation and rounding error analysis are required to ensure that sensible results can still be obtained. In this paper we study the accumulation of rounding errors in the solution of the heat equation, a proxy for parabolic PDEs, via Runge-Kutta finite difference methods using round-to-nearest (RtN) and stochastic rounding (SR). We demonstrate how to implement the scheme to reduce rounding errors and we derive \emph{a priori} estimates for local and global rounding errors. Let $u$ be the unit roundoff. While the worst-case local errors are $O(u)$ with respect to the discretization parameters (mesh size and timestep), the RtN and SR error behavior is substantially different. In fact, the RtN solution always stagnates for small enough $\Delta t$, and until stagnation the global error grows like $O(u\Delta t^{-1})$. In contrast, we show that the leading-order errors introduced by SR are zero-mean, independent in space and mean-independent in time, making SR resilient to stagnation and rounding error accumulation. In fact, we prove that for SR the global rounding errors are only $O(u\Delta t^{-1/4})$ in 1D and are essentially bounded (up to logarithmic factors) in higher dimensions.