A closed loop gradient descent algorithm applied to Rosenbrock's function

TL;DR

This paper presents a new adaptive damping technique for an inertial gradient system, improving optimization on Rosenbrock's function through continuous and discrete-time analyses with stability and simulation results.

Contribution

It introduces a novel adaptive damping method for inertial gradient systems and demonstrates its effectiveness on Rosenbrock's function with stability analysis and numerical simulations.

Findings

Improved optimization performance on Rosenbrock's function.

Lyapunov stability demonstrated for the continuous-time algorithm.

Discrete-time implementation shows promising results.

Abstract

We introduce a novel adaptive damping technique for an inertial gradient system which finds application as a gradient descent algorithm for unconstrained optimisation. In an example using the non-convex Rosenbrock's function, we show an improvement on existing momentum-based gradient optimisation methods. Also using Lyapunov stability analysis, we demonstrate the performance of the continuous-time version of the algorithm. Using numerical simulations, we consider the performance of its discrete-time counterpart obtained by using the symplectic Euler method of discretisation.