Follow the Clairvoyant: an Imitation Learning Approach to Optimal Control

TL;DR

This paper introduces a novel imitation learning approach for optimal control that directly minimizes tracking error relative to the clairvoyant policy, leading to improved performance over regret minimization methods, especially under constraints.

Contribution

It proposes an efficient optimization-based method for follow-the-clairvoyant control, bridging classical control laws and outperforming regret minimization in constrained scenarios.

Findings

The approach attains minimal regret without constraints.

Numerical experiments show superior tracking of clairvoyant trajectories.

Method interpolates between H2 and H∞ control laws.

Abstract

We consider control of dynamical systems through the lens of competitive analysis. Most prior work in this area focuses on minimizing regret, that is, the loss relative to an ideal clairvoyant policy that has noncausal access to past, present, and future disturbances. Motivated by the observation that the optimal cost only provides coarse information about the ideal closed-loop behavior, we instead propose directly minimizing the tracking error relative to the optimal trajectories in hindsight, i.e., imitating the clairvoyant policy. By embracing a system level perspective, we present an efficient optimization-based approach for computing follow-the-clairvoyant (FTC) safe controllers. We prove that these attain minimal regret if no constraints are imposed on the noncausal benchmark. In addition, we present numerical experiments to show that our policy retains the hallmark of competitive algorithms of interpolating between classical $\mathcal{H}_2$ and $\mathcal{H}_\infty$ control laws - while consistently outperforming regret minimization methods in constrained scenarios thanks to the superior ability to chase the clairvoyant.