Preconditioned Low-Rank Riemannian Optimization for Symmetric Positive Definite Linear Matrix Equations

TL;DR

This paper develops a preconditioned Riemannian conjugate gradient method for efficiently solving large-scale symmetric positive definite matrix equations, especially when the solution admits low-rank approximations, improving over existing approaches.

Contribution

It introduces novel preconditioning strategies within Riemannian optimization for low-rank solutions to matrix equations, extending methods for cases with more than two terms.

Findings

The proposed method is competitive on practical examples.

New preconditioning techniques improve convergence.

Adaptive rank strategies enhance efficiency.

Abstract

This work is concerned with the numerical solution of large-scale symmetric positive definite matrix equations of the form $A_1XB_1^\top + A_2XB_2^\top + \dots + A_\ell X B_\ell^\top = F$, as they arise from discretized partial differential equations and control problems. One often finds that $X$ admits good low-rank approximations, in particular when the right-hand side matrix $F$ has low rank. For $\ell \le 2$ terms, the solution of such equations is well studied and effective low-rank solvers have been proposed, including Alternating Direction Implicit (ADI) methods for Lyapunov and Sylvester equations. For $\ell > 2$, several existing methods try to approach $X$ through combining a classical iterative method, such as the conjugate gradient (CG) method, with low-rank truncation. In this work, we consider a more direct approach that approximates $X$ on manifolds of fixed-rank matrices through Riemannian CG. One particular challenge is the incorporation of effective preconditioners into such a first-order Riemannian optimization method. We propose several novel preconditioning strategies, including a change of metric in the ambient space, preconditioning the Riemannian gradient, and a variant of ADI on the tangent space. Combined with a strategy for adapting the rank of the approximation, the resulting method is demonstrated to be competitive for a number of examples representative for typical applications.