Suboptimality analysis of receding horizon quadratic control with unknown linear systems and its applications in learning-based control

TL;DR

This paper investigates the factors affecting the performance of receding-horizon quadratic control in unknown linear systems, providing bounds and insights for improving learning-based control strategies.

Contribution

It introduces a novel perturbation analysis of the Riccati equation, deriving performance bounds and linking prediction horizon choices to controllability and learning guarantees.

Findings

Optimal prediction horizon can be one or infinity depending on errors.

Finite horizon larger than controllability index suffices for near-optimality.

Adaptive horizon increasing logarithmically improves regret in learning control.

Abstract

This work analyzes how the trade-off between the modeling error, the terminal value function error, and the prediction horizon affects the performance of a nominal receding-horizon linear quadratic (LQ) controller. By developing a novel perturbation result of the Riccati difference equation, a novel performance upper bound is obtained and suggests that for many cases, the prediction horizon can be either one or infinity to improve the control performance, depending on the relative difference between the modeling error and the terminal value function error. The result also shows that when an infinite horizon is desired, a finite prediction horizon that is larger than the controllability index can be sufficient for achieving a near-optimal performance, revealing a close relation between the prediction horizon and controllability. The obtained suboptimality performance upper bound is applied to provide novel sample complexity and regret guarantees for nominal receding-horizon LQ controllers in a learning-based setting. We show that an adaptive prediction horizon that increases as a logarithmic function of time is beneficial for regret minimization.