A subspace-conjugate gradient method for linear matrix equations

TL;DR

This paper introduces a novel subspace-conjugate gradient method for efficiently solving large-scale symmetric positive definite linear matrix equations, leveraging low-rank formats and randomized strategies to improve memory efficiency and convergence.

Contribution

The paper presents a new iterative algorithm that advances existing methods like truncated matrix-oriented CG by exploiting subspace information and incorporating memory-saving techniques.

Findings

The new method outperforms existing approaches in large-scale applications.

It effectively manages memory constraints using randomized range-finding.

Experimental results demonstrate improved convergence and efficiency.

Abstract

The efficient solution of large-scale multiterm linear matrix equations is a challenging task in numerical linear algebra, and it is a largely open problem. We propose a new iterative scheme for symmetric and positive definite operators, significantly advancing methods such as truncated matrix-oriented Conjugate Gradients (CG). The new algorithm capitalizes on the low-rank matrix format of its iterates by fully exploiting the subspace information of the factors as iterations proceed. The approach implicitly relies on orthogonality conditions imposed over much larger subspaces than in CG, unveiling insightful connections with subspace projection methods. The new method is also equipped with memory-saving strategies. In particular, we show that for a given matrix $\mathbf{Y}$, the action $\mathcal{L}(\mathbf{Y})$ in low rank format may not be evaluated exactly due to memory constraints. This problem is often underestimated, though it will eventually produce Out-of-Memory breakdowns for a sufficiently large number of terms. We propose an ad-hoc randomized range-finding strategy that appears to fully resolve this shortcoming. Experimental results with typical application problems illustrate the potential of our approach over various methods developed in the recent literature.