The turnpike property and the long-time behavior of the Hamilton-Jacobi-Bellman equation for finite-dimensional LQ control problems

TL;DR

This paper investigates the long-term behavior of the value function in finite-dimensional LQ control problems, revealing a specific asymptotic form linked to the turnpike property and extending understanding to constrained controls and non-coercive Hamiltonians.

Contribution

It establishes the asymptotic form of the value function as the time horizon grows, linking it to the turnpike property, and analyzes the long-time behavior of the Hamilton-Jacobi-Bellman equation under constraints.

Findings

Value function asymptotically behaves as W(x) + c T + λ as T→∞

Turnpike property holds even with control constraints and saturation

Long-time behavior of HJB solutions with non-coercive Hamiltonian

Abstract

We analyze the consequences that the so-called turnpike property has on the long-time behavior of the value function corresponding to a finite-dimensional linear-quadratic optimal control problem with general terminal cost and constrained controls. We prove that, when the time horizon $T$ tends to infinity, the value function asymptotically behaves as $W(x) + c\, T + \lambda$, and we provide a control interpretation of each of these three terms, making clear the link with the turnpike property. As a by-product, we obtain the long-time behavior of the solution to the associated Hamilton-Jacobi-Bellman equation in a case where the Hamiltonian is not coercive in the momentum variable. As a result of independent interest, we showed that linear-quadratic optimal control problems with constrained control enjoy a turnpike property, also particularly when the steady optimum may saturate the control constraints.