Regularized dynamical parametric approximation

TL;DR

This paper develops a regularized approach for numerically approximating evolution equations using nonlinear parametrizations, addressing irregularities in the parametrization map and demonstrating its effectiveness through theoretical case studies and numerical experiments.

Contribution

It introduces a novel regularization technique for nonlinear parametric approximation of evolution equations, handling irregular parametrizations and analyzing the interplay between regularization and discretization.

Findings

Effective regularization in irregular parametrizations

Successful application to quantum dynamics and neural networks

Theoretical and numerical validation of the approach

Abstract

This paper studies the numerical approximation of evolution equations by nonlinear parametrizations $u(t)=\Phi(\param(t))$ with time-dependent parameters $\param(t)$, which are to be determined in the computation. The motivation comes from approximations in quantum dynamics by multiple Gaussians and approximations of various dynamical problems by tensor networks and neural networks. In all these cases, the parametrization is typically irregular: the derivative $\Phi'(\param)$ can have arbitrarily small singular values and may have varying rank. We derive approximation results for a regularized approach in the time-continuous case as well as in time-discretized cases. For the latter, there is a nontrivial interplay between the regularization parameter and the time stepsize, depending also on the defect size and local bounds of the second derivative of the parametrization map $\Phi$. When this is appropriately taken into account, the approach can be successfully applied in irregular situations, even though it runs counter to the basic principle in numerical analysis to avoid solving ill-posed subproblems when aiming for a stable algorithm. The paper also includes two theoretical case studies: regularized parametric steepest descent for optimization and regularized parametric Strang splitting for the time-dependent Schr\"odinger equation. Numerical experiments with sums of Gaussians for approximating quantum dynamics and with neural networks for approximating the flow map of a system of ordinary differential equations illustrate and complement the theoretical results.