Preconditioned accelerated gradient descent methods for locally Lipschitz smooth objectives with applications to the solution of nonlinear PDEs

TL;DR

This paper establishes a theoretical foundation for preconditioned accelerated gradient descent (PAGD) in solving nonlinear PDEs, demonstrating exponential convergence and improved efficiency over traditional methods through rigorous analysis and numerical experiments.

Contribution

It introduces a new theoretical framework for PAGD applied to PDEs, including invariant set existence, convergence rates, and a connection to a second-order ODE, with practical numerical validation.

Findings

PAGD achieves exponential convergence for locally Lipschitz smooth objectives.

The method's convergence rate is mesh size-independent and faster than PGD.

Numerical experiments confirm the theoretical acceleration and stability.

Abstract

We develop a theoretical foundation for the application of Nesterov's accelerated gradient descent method (AGD) to the approximation of solutions of a wide class of partial differential equations (PDEs). This is achieved by proving the existence of an invariant set and exponential convergence rates when its preconditioned version (PAGD) is applied to minimize locally Lipschitz smooth, strongly convex objective functionals. We introduce a second-order ordinary differential equation (ODE) with a preconditioner built-in and show that PAGD is an explicit time-discretization of this ODE, which requires a natural time step restriction for energy stability. At the continuous time level, we show an exponential convergence of the ODE solution to its steady state using a simple energy argument. At the discrete level, assuming the aforementioned step size restriction, the existence of an invariant set is proved and a matching exponential rate of convergence of the PAGD scheme is derived by mimicking the energy argument and the convergence at the continuous level. Applications of the PAGD method to numerical PDEs are demonstrated with certain nonlinear elliptic PDEs using pseudo-spectral methods for spatial discretization, and several numerical experiments are conducted. The results confirm the global geometric and mesh size-independent convergence of the PAGD method, with an accelerated rate that is improved over the preconditioned gradient descent (PGD) method.