LP Based Upper and Lower Bounds for Ces\`aro and Abel Limits of the Optimal Values in Problems of Control of Stochastic Discrete Time Systems

TL;DR

This paper investigates the asymptotic behavior of stochastic control problems in discrete time, showing that Cesàro and Abel limits of optimal values can be characterized via infinite-dimensional linear programming and its dual.

Contribution

It introduces a novel approach to evaluate Cesàro and Abel limits of optimal values using infinite-dimensional linear programming for stochastic control problems.

Findings

Cesàro and Abel limits are characterized by linear programming formulations.

The approach applies to Markov decision processes with time averaging and discounting.

Provides a new method for analyzing long-term behavior of stochastic control systems.

Abstract

In this paper, we study asymptotic properties of problems of control of stochastic discrete time systems (also known as Markov decision processes) with time averaging and time discounting optimality criteria, and we establish that the Ces\`aro and Abel limits of the optimal values in such problems can be evaluated with the help of a certain infinite-dimensional linear programming problem and its dual.