Discounted LQR: stabilizing (near-)optimal state-feedback laws

TL;DR

This paper investigates stability conditions for discounted LQR problems, proposing new criteria and methods to design stabilizing near-optimal controllers when standard approaches fail due to small discount factors.

Contribution

It introduces novel necessary and sufficient stability conditions based on the optimal value function and develops LMI-based and policy iteration methods for stabilizing near-optimal controllers.

Findings

New stability conditions improve existing criteria.

LMI-based conditions enable controller synthesis.

Numerical examples demonstrate effectiveness.

Abstract

We study deterministic, discrete linear time-invariant systems with infinite-horizon discounted quadratic cost. It is well-known that standard stabilizability and detectability properties are not enough in general to conclude stability properties for the system in closed-loop with the optimal controller when the discount factor is small. In this context, we first review some of the stability conditions based on the optimal value function found in the learning and control literature and highlight their conservatism. We then propose novel (necessary and) sufficient conditions, still based on the optimal value function, under which stability of the origin for the optimal closed-loop system is guaranteed. Afterwards, we focus on the scenario where the optimal feedback law is not stabilizing because of the discount factor and the goal is to design an alternative stabilizing near-optimal static state-feedback law. We present both linear matrix inequality-based conditions and a variant of policy iteration to construct such stabilizing near-optimal controllers. The methods are illustrated via numerical examples.