Structured Output Feedback Control for Linear Quadratic Regulator Using Policy Gradient Method

TL;DR

This paper develops a model-free policy gradient method with variance reduction for structured output feedback control in LQR problems, demonstrating global convergence and efficiency in numerical experiments.

Contribution

It introduces a novel policy gradient algorithm with variance reduction for structured output feedback LQR, ensuring global convergence in a model-free setting.

Findings

The algorithm converges globally to ε-stationary points.

Variance reduction significantly improves gradient estimation.

Numerical results show efficient problem-solving in practice.

Abstract

We consider the static output feedback control for Linear Quadratic Regulator problems with structured constraints under the assumption that system parameters are unknown. To solve the problem in the model free setting, we propose the policy gradient algorithm based on the gradient projection method and show its global convergence to $\varepsilon$-stationary points. In addition, we introduce a variance reduction technique and show both theoretically and numerically that it significantly reduces the variance in the gradient estimation. We also show in the numerical experiments that the model free approach efficiently solves the problem.