An Empirical Study of Conjugate Gradient Preconditioners for Solving Symmetric Positive Definite Systems of Linear Equations

TL;DR

This study benchmarks 79 matrices with 10 preconditioners for symmetric positive definite systems, revealing that incomplete symmetric factorizations like IC often outperform classical methods and direct solvers, with performance influenced by ordering strategies.

Contribution

It provides a comprehensive performance comparison of common preconditioners for SPD systems, highlighting the effectiveness of incomplete factorizations and the impact of ordering strategies.

Findings

Incomplete symmetric factorizations like IC are often most effective.

Multigrid methods can perform exceptionally well in some cases.

Simple classical techniques like symmetric Gauss Seidel are generally not productive.

Abstract

Despite hundreds of papers on preconditioned linear systems of equations, there remains a significant lack of comprehensive performance benchmarks comparing various preconditioners for solving symmetric positive definite (SPD) systems. In this paper, we present a comparative study of 79 matrices using a broad range of preconditioners. Specifically, we evaluate 10 widely used preconditoners across 108 configurations to assess their relative performance against using no preconditioner. Our focus is on preconditioners that are commonly used in practice, are available in major software packages, and can be utilized as black-box tools without requiring significant \textit{a priori} knowledge. In addition, we compare these against a selection of classical methods. We primarily compare them without regards to effort needed to compute the preconditioner. Our results show that symmetric positive definite systems are mostly likely to benefit from incomplete symmetric factorizations, such as incomplete Cholesky (IC). Multigrid methods occasionally do exceptionally well. Simple classical techniques, symmetric Gauss Seidel and symmetric SOR, are not productive. We find that including preconditioner construction costs significantly diminishes the advantages of iterative methods compared to direct solvers; although, tuned IC methods often still outperform direct methods. Additionally, ordering strategies such as approximate minimum degree significantly enhance IC effectiveness. We plan to expand the benchmark with larger matrices, additional solvers, and detailed metrics to provide actionable information on SPD preconditioning.