Average-Cost MDPs with Infinite State and Action Sets: New Sufficient Conditions for Optimality Inequalities and Equations

TL;DR

This paper establishes new sufficient conditions for the validity of optimality inequalities and equations in infinite-horizon average-cost MDPs with infinite state and action spaces, ensuring the existence of deterministic optimal policies.

Contribution

It introduces novel conditions for optimality in average-cost MDPs with continuous transition probabilities, expanding theoretical understanding.

Findings

New sufficient conditions for optimality inequalities

Conditions ensuring the validity of optimality equations

Existence of deterministic optimal policies under these conditions

Abstract

This paper studies discrete-time average-cost infinite-horizon Markov decision processes (MDPs) with Borel state and action sets. It introduces new sufficient conditions for { the} validity of optimality inequalities and optimality equations for MDPs with weakly and setwise continuous transition probabilities. These inequalities and equations imply the existence of deterministic optimal policies.