Split optimal policy iteration for LQR problems

TL;DR

This paper analyzes the convergence properties of split optimal policy iteration for coupled LQR problems, revealing different convergence rates for continuous and discrete time systems, with quadratic convergence in continuous time and linear in discrete time.

Contribution

It provides a detailed analysis of the convergence behavior of split optimal policy iteration in coupled LQR problems, highlighting differences between continuous and discrete systems.

Findings

Global convergence for both continuous and discrete systems

Quadratic local convergence in continuous time

Linear local convergence in discrete time

Abstract

This technical report is concerned with the convergence properties of what we call the split optimal policy iteration for coupled LQR problems; see section 3.1 in the manuscript. Interestingly, the iteration shows different convergence behavior for continuous and discrete time systems: while global convergence holds for both cases, we have local quadratic convergence for the continuous time case, but only linear convergence for the discrete time case - even though quadratic convergence is retained in the limit as the coupling between the subsystems vanishes.