Characterizing the learning dynamics in extremum seeking

TL;DR

This paper analyzes the learning dynamics of extremum seeking control, revealing it approximates an averaged gradient of the objective function, which impacts the understanding of local extrema and global optimization strategies.

Contribution

It provides an explicit quantification of the recovered gradient in extremum seeking and introduces a new perspective on global optimization through averaging effects.

Findings

Recovered gradient corresponds to an averaged gradient of the objective function.

Extremum seeking may optimize a different function than the original.

Local extrema can be smoothed out due to averaging in the learning dynamics.

Abstract

We consider perturbation-based extremum seeking, which recovers an approximate gradient of an analytically unknown objective function through measurements. Using classical needle variation analysis, we are able to explicitly quantify the recovered gradient in the scalar case. We reveal that it corresponds to an averaged gradient of the objective function, even for very general extremum seeking systems. From this, we create a recursion which represents the learning dynamics along the recovered gradient. These results give rise to the interpretation that extremum seeking actually optimizes a function other than the original one. From this insight, a new perspective on global optimization of functions with local extrema emerges: because the gradient is averaged over a certain time period, local extrema might be evened out in the learning dynamics. Moreover, a multidimensional extension of the scalar results is given.