Extremum Seeking Optimal Controls of Unknown Systems

TL;DR

This paper introduces an extremum seeking control method for unknown, time-varying systems that can be repeatedly re-initialized, enabling near-optimal control despite noise and system uncertainties.

Contribution

It proposes a novel extremum seeking algorithm that guarantees convergence to minimal cost controllers for unknown systems with convex cost functions.

Findings

Algorithm approaches minimal cost in simulations

Effective in noisy, time-varying systems

Applicable to industrial processes with repeated operations

Abstract

We present a method for finding optimal controllers for unknown, time-varying, dynamic systems which can be re-initialized from a given initial condition repeatedly, in which the performance measure is available for sampling with noise, but analytically unknown. Such systems are present throughout industry where processes must be repeated many times, such as a voltage source which is repeatedly turned on for a fraction of a second from zero initial conditions and then turned off again, whose output must track a specific trajectory, while the system's components are slowly drifting with time due to temperature variations. For systems with convex cost functions we prove that our algorithm will produce controllers that approach the minimal cost, e.g., the cost minimizing LQR optimal controller that could have been designed analytically had the system and objective function been known. We demonstrate the algorithm's effectiveness with simulation studies of noisy and time-varying systems.