Gradient Dominance in the Linear Quadratic Regulator: A Unified Analysis for Continuous-Time and Discrete-Time Systems

TL;DR

This paper establishes a unified gradient dominance property for both continuous-time and discrete-time LQRs, simplifying analysis and revealing structural similarities, which enables linear convergence guarantees for policy optimization.

Contribution

It introduces a unified proof framework for gradient dominance in LQRs applicable to both time models, based on a convex reformulation and common Lyapunov inequalities.

Findings

Unified gradient dominance property for continuous and discrete LQRs

Clarifies structural symmetry between time models

Supports theoretical results with numerical examples

Abstract

Despite its nonconvexity, policy optimization for the Linear Quadratic Regulator (LQR) admits a favorable structural property known as gradient dominance, which facilitates linear convergence of policy gradient methods to the globally optimal gain. While gradient dominance has been extensively studied, continuous-time and discrete-time LQRs have largely been analyzed separately, relying on slightly different assumptions, proof strategies, and resulting guarantees. In this paper, we present a unified gradient dominance property for both continuous-time and discrete-time LQRs under mild stabilizability and detectability assumptions. Our analysis is based on a convex reformulation derived from a common Lyapunov inequality representation and a unified change-of-variables procedure. This convex-lifting perspective yields a single proof framework applicable to both time models. The unified treatment clarifies how differences between continuous-time and discrete-time dynamics influence theoretical guarantees and reveals a deeper structural symmetry between the two formulations. Numerical examples illustrate and support the theoretical findings.