Mean-Variance Type Controls Involving a Hidden Markov Chain: Models and Numerical Approximation

TL;DR

This paper develops a numerical approximation method for mean-variance control problems involving hidden Markov chains with noisy observations, applicable to networked systems, and demonstrates convergence with a numerical example.

Contribution

It introduces a Markov chain approximation scheme for partially observed mean-variance control problems with hidden Markov chains, including convergence analysis.

Findings

Convergence of the approximation algorithm is established.

Numerical example demonstrates the effectiveness of the method.

Abstract

Motivated by applications arising in networked systems, this work examines controlled regime-switching systems that stem from a mean-variance formulation. A main point is that the switching process is a hidden Markov chain. An additional piece of information, namely, a noisy observation of switching process corrupted by white noise is available. We focus on minimizing the variance subject to a fixed terminal expectation. Using the Wonham filter, we convert the partially observed system to a completely observable one first. Since closed-form solutions are virtually impossible be obtained, a Markov chain approximation method is used to devise a computational scheme. Convergence of the algorithm is obtained. A numerical example is provided to demonstrate the results.