Cautious Bayesian MPC: Regret Analysis and Bounds on the Number of Unsafe Learning Episodes

TL;DR

This paper introduces Cautious Bayesian MPC, a method combining MPC and posterior sampling with constraint tightening to ensure safety and provide regret bounds during exploration in control tasks.

Contribution

It proposes a simple constraint tightening technique for cautious exploration in Bayesian MPC, with theoretical regret analysis and safety guarantees for linear and nonlinear systems.

Findings

Provides regret bounds for the proposed method.

Shows bound on unsafe episodes during learning.

Demonstrates effectiveness through numerical examples.

Abstract

This paper investigates the combination of model predictive control (MPC) concepts and posterior sampling techniques and proposes a simple constraint tightening technique to introduce cautiousness during explorative learning episodes. The provided theoretical analysis in terms of cumulative regret focuses on previously stated sufficient conditions of the resulting `Cautious Bayesian MPC' algorithm and shows Lipschitz continuity of the future reward function in the case of linear MPC problems. In the case of nonlinear MPC problems, it is shown that commonly required assumptions for nonlinear MPC optimization techniques provide sufficient criteria for model-based RL using posterior sampling. Furthermore, it is shown that the proposed constraint tightening implies a bound on the expected number of unsafe learning episodes in the linear and nonlinear case using a soft-constrained MPC formulation. The efficiency of the method is illustrated using numerical examples.