Low-rank plus diagonal approximations for Riccati-like matrix differential equations

TL;DR

This paper introduces a low-rank plus diagonal approximation method for large positive semi-definite matrices evolving in time, enabling efficient computation and invertibility for Riccati-like differential equations in high dimensions.

Contribution

It proposes a novel low-rank plus diagonal approximation framework with a closed-form projection formula, improving tractability and invertibility over previous low-rank methods.

Findings

Efficient linear-cost computation for high-dimensional Riccati equations.

Enhanced invertibility of approximations due to the diagonal component.

Successful application to Bayesian inference and Kalman filtering.

Abstract

We consider the problem of computing tractable approximations of time-dependent d x d large positive semi-definite (PSD) matrices defined as solutions of a matrix differential equation. We propose to use "low-rank plus diagonal" PSD matrices as approximations that can be stored with a memory cost being linear in the high dimension d. To constrain the solution of the differential equation to remain in that subset, we project the derivative at all times onto the tangent space to the subset, following the methodology of dynamical low-rank approximation. We derive a closed-form formula for the projection, and show that after some manipulations it can be computed with a numerical cost being linear in d, allowing for tractable implementation. Contrary to previous approaches based on pure low-rank approximations, the addition of the diagonal term allows for our approximations to be invertible matrices, that can moreover be inverted with linear cost in d. We apply the technique to Riccati-like equations, then to two particular problems. Firstly a low-rank approximation to our recent Wasserstein gradient flow for Gaussian approximation of posterior distributions in approximate Bayesian inference, and secondly a novel low-rank approximation of the Kalman filter for high-dimensional systems. Numerical simulations illustrate the results.