Sequential learning and control: Targeted exploration for robust performance

TL;DR

This paper introduces a dual control strategy combining targeted harmonic exploration and gain-scheduling to improve robust performance in uncertain linear systems, with probabilistic guarantees and optimized exploration.

Contribution

It proposes a novel sequential approach that optimizes exploration for robust control, using spectral line excitation bounds and semidefinite programming for gain-scheduling design.

Findings

Enhanced exploration with finite excitation bounds

Probabilistic performance guarantees for control

Demonstrated effectiveness on a hard-to-learn system

Abstract

We present a novel dual control strategy for uncertain linear systems based on targeted harmonic exploration and gain-scheduling with performance and excitation guarantees. In the proposed sequential approach, robust control is implemented after exploration with the main feature that the exploration is optimized with respect to the robust control performance. Specifically, we leverage recent results on finite excitation using spectral lines to determine a high probability lower bound on the resultant finite excitation of the exploration data. This provides an a priori upper bound on the remaining model uncertainty after exploration, which can further be leveraged in a gain-scheduling controller design that guarantees robust performance. This leads to a semidefinite program-based design which computes an exploration strategy with finite excitation bounds and minimal energy, and a gain-scheduled controller with probabilistic performance bounds that can be implemented after exploration. The effectiveness of our approach and its benefits over common random exploration strategies are demonstrated with an example of a system which is 'hard to learn'.