Policy Iteration for Multiplicative Noise Output Feedback Control

TL;DR

This paper introduces a policy iteration algorithm for multiplicative noise LQ output feedback control, demonstrating faster convergence than previous methods and opening avenues for data-driven POMDP solutions.

Contribution

The paper presents a novel policy iteration algorithm for multiplicative noise LQ control, solving coupled Riccati equations efficiently and outperforming existing value iteration methods.

Findings

Faster convergence than value iteration algorithm

Effective solution to coupled Riccati equations in POMDPs

Potential for data-driven control approaches

Abstract

We propose a policy iteration algorithm for solving the multiplicative noise linear quadratic output feedback design problem. The algorithm solves a set of coupled Riccati equations for estimation and control arising from a partially observable Markov decision process (POMDP) under a class of linear dynamic control policies. We show in numerical experiments far faster convergence than a value iteration algorithm, formerly the only known algorithm for solving this class of problem. The results suggest promising future research directions for policy optimization algorithms in more general POMDPs, including the potential to develop novel approximate data-driven approaches when model parameters are not available.