Linear Convergence of Data-Enabled Policy Optimization for Linear Quadratic Tracking

TL;DR

This paper extends the Data-enabled Policy Optimization (DeePO) framework to linear quadratic tracking (LQT) with offline data, demonstrating linear convergence and validating the approach through numerical experiments.

Contribution

It introduces a covariance parameterization for LQT, derives a data-driven formulation, and proves linear convergence of DeePO for LQT, bridging model-based and data-driven methods.

Findings

Proves linear convergence of DeePO for LQT.

Develops a covariance-based data-driven LQT formulation.

Validates convergence with numerical experiments.

Abstract

Data-enabled policy optimization (DeePO) is a newly proposed method to attack the open problem of direct adaptive LQR. In this work, we extend the DeePO framework to the linear quadratic tracking (LQT) with offline data. By introducing a covariance parameterization of the LQT policy, we derive a direct data-driven formulation of the LQT problem. Then, we use gradient descent method to iteratively update the parameterized policy to find an optimal LQT policy. Moreover, by revealing the connection between DeePO and model-based policy optimization, we prove the linear convergence of the DeePO iteration. Finally, a numerical experiment is given to validate the convergence results. We hope our work paves the way to direct adaptive LQT with online closed-loop data.