On the Guarantees of Minimizing Regret in Receding Horizon

TL;DR

This paper introduces a receding horizon control scheme that minimizes regret to bridge optimal control and online learning, providing stability, safety, and efficiency guarantees.

Contribution

It presents the first receding horizon approach based on finite horizon regret-optimal policies with proven stability, safety, and computational efficiency.

Findings

Outperforms standard methods when disturbances are unpredictable.

Ensures recursive feasibility and bounded regret.

Efficient convex-concave programming for policy optimization.

Abstract

Towards bridging classical optimal control and online learning, regret minimization has recently been proposed as a control design criterion. This competitive paradigm penalizes the loss relative to the optimal control actions chosen by a clairvoyant policy, and allows tracking the optimal performance in hindsight no matter how disturbances are generated. In this paper, we propose the first receding horizon scheme based on the repeated computation of finite horizon regret-optimal policies, and we establish stability and safety guarantees for the resulting closed-loop system. Our derivations combine novel monotonicity properties of clairvoyant policies with suitable terminal ingredients. We prove that our scheme is recursively feasible, stabilizing, and that it achieves bounded regret relative to the infinite horizon clairvoyant policy. Last, we show that the policy optimization problem can be solved efficiently through convex-concave programming. Our numerical experiments show that minimizing regret can outperform standard receding horizon approaches when the disturbances poorly fit classical design assumptions - even when the finite horizon planning is recomputed less frequently.