# On Linear Programming for Constrained and Unconstrained Average-Cost Markov Decision Processes with Countable Action Spaces and Strictly Unbounded Costs

## TL;DR

This paper develops a linear programming framework for average-cost Markov decision processes with countable actions and unbounded costs, proving duality and optimality without requiring lower-semicontinuity.

## Contribution

It introduces a novel approach that handles discontinuous dynamics and costs in countable action space MDPs using a strict unboundedness condition and a majorization condition.

## Key findings

- No duality gap in the linear programming formulation.
- Applicable to discontinuous MDP models.
- Proven optimality results for a broad class of MDPs.

## Abstract

We consider the linear programming approach for constrained and unconstrained Markov decision processes (MDPs) under the long-run average cost criterion, where the class of MDPs in our study have Borel state spaces and discrete countable action spaces. Under a strict unboundedness condition on the one-stage costs and a recently introduced majorization condition on the state transition stochastic kernel, we study infinite-dimensional linear programs for the average-cost MDPs and prove the absence of a duality gap and other optimality results. Our results do not require a lower-semicontinuous MDP model. Thus, they can be applied to countable action space MDPs where the dynamics and one-stage costs are discontinuous in the state variable. Our proofs make use of the continuity property of Borel measurable functions asserted by Lusin's theorem.