Continuity of Filters for Discrete-Time Control Problems Defined by Explicit Equations

TL;DR

This paper establishes conditions ensuring weak continuity of filters in discrete-time stochastic control problems, which guarantees the existence of optimal policies and convergence of value iteration.

Contribution

It provides new criteria for weak continuity of transition probabilities in filters, aiding the analysis of optimal control in partially observable systems.

Findings

Weak continuity of filters is guaranteed under specified conditions.

Continuity in total variation of transition probabilities is established.

Applications demonstrate the practical relevance of the theoretical results.

Abstract

Discrete time control systems whose dynamics and observations are described by stochastic equations are common in engineering, operations research, health care, and economics. For example, stochastic filtering problems are usually defined via stochastic equations. These problems can be reduced to Markov decision processes (MDPs) whose states are posterior state distributions, and transition probabilities for such MDPs are sometimes called filters. This paper investigates sufficient conditions on transition and observation functions for the original problems to guarantee weak continuity of the filter. Under mild conditions on cost functions, weak continuity implies the existence of optimal policies minimizing the expected total costs, the validity of optimality equations, and convergence of value iterations to optimal values. This paper uses recent results on weak continuity of filters for partially observable MDPs defined by transition and observation probabilities. It develops a criterion of weak continuity of transition probabilities and a sufficient condition for continuity in total variation of transition probabilities. The results are illustrated with applications to filtering problems.