Fatou's Lemma in Its Classic Form and Lebesgue's Convergence Theorems for Varying Measures with Applications to MDPs

TL;DR

This paper extends Fatou's lemma and Lebesgue's convergence theorems to sequences of measures with different convergence modes, providing conditions for their classic forms and applications to Markov decision processes.

Contribution

It establishes sufficient conditions for Fatou's lemma to hold in its classic form under weak and setwise convergence of measures, and applies these results to MDPs.

Findings

Sufficient conditions for Fatou's lemma in its classic form under weak convergence.

Lebesgue's and monotone convergence theorems for varying measures.

Broad conditions for optimality equations in average-cost MDPs.

Abstract

The classic Fatou lemma states that the lower limit of a sequence of integrals of functions is greater or equal than the integral of the lower limit. It is known that Fatou's lemma for a sequence of weakly converging measures states a weaker inequality because the integral of the lower limit is replaced with the integral of the lower limit in two parameters, where the second parameter is the argument of the functions. This paper provides sufficient conditions when Fatou's lemma holds in its classic form for a sequence of weakly converging measures. The functions can take both positive and negative values. The paper also provides similar results for sequences of setwise converging measures. It also provides Lebesgue's and monotone convergence theorems for sequences of weakly and setwise converging measures. The obtained results are used to prove broad sufficient conditions for the validity of optimality equations for average-cost Markov decision processes.