Gradient approximation and extremum seeking via needle variations

TL;DR

This paper introduces a gradient approximation method using needle-shaped inputs, deriving insights from Pontryagin's Maximum Principle, and proposes a new optimization algorithm combining heavy ball and Nesterov's methods.

Contribution

It presents a novel gradient approximation scheme based on needle variations and develops a new optimization algorithm inspired by these ideas.

Findings

System moves along a weighted averaged gradient

Applicable to arbitrary periodic inputs

New gradient-based optimization algorithm proposed

Abstract

We consider a gradient approximation scheme that is based on applying needle shaped inputs. By using ideas known from the classic proof of the Pontryagin Maximum Principle we derive an approximation that reveals that the considered system moves along a weighted averaged gradient. Moreover, based on the same ideas, we give similar results for arbitrary periodic inputs. We also present a new gradient-based optimization algorithm that is motivated by our calculations and that can be interpreted as a combination of the heavy ball method and Nesterov's method.