Average-case analysis of the Gaussian Elimination with Partial Pivoting

TL;DR

This paper provides a probabilistic analysis showing that Gaussian Elimination with Partial Pivoting (GEPP) is typically stable for random Gaussian matrices, with the growth factor being polynomially bounded in size, improving previous bounds.

Contribution

The paper offers a partial theoretical justification for GEPP's typical stability by proving polynomial bounds on the growth factor for random Gaussian matrices, refining earlier estimates.

Findings

Growth factor is at most polynomial in size with high probability.

GEPP requires only logarithmic additional bits of precision for accuracy.

Tail estimates support empirical stability observations.

Abstract

The Gaussian Elimination with Partial Pivoting (GEPP) is a classical algorithm for solving systems of linear equations. Although in specific cases the loss of precision in GEPP due to roundoff errors can be very significant, empirical evidence strongly suggests that for a {\it typical} square coefficient matrix, GEPP is numerically stable. We obtain a (partial) theoretical justification of this phenomenon by showing that, given the random $n\times n$ standard Gaussian coefficient matrix $A$, the {\it growth factor} of the Gaussian Elimination with Partial Pivoting is at most polynomially large in $n$ with probability close to one. This implies that with probability close to one the number of bits of precision sufficient to solve $Ax = b$ to $m$ bits of accuracy using GEPP is $m+O(\log n)$, which improves an earlier estimate $m+O(\log^2 n)$ of Sankar, and which we conjecture to be optimal by the order of magnitude. We further provide tail estimates of the growth factor which can be used to support the empirical observation that GEPP is more stable than the Gaussian Elimination with no pivoting.