Existence of Markov equilibrium control in discrete time

TL;DR

This paper establishes conditions for the existence of Markov equilibrium controls in discrete-time, finite-horizon stochastic control problems with time-inconsistency, addressing an open problem in the field.

Contribution

It provides a sufficient condition for the existence of Markov equilibrium policies when costs are lower semicontinuous and transition kernels are continuous, even without compact control spaces.

Findings

Existence of Markov equilibrium control under specified conditions.

Sufficient conditions include lower semicontinuity and boundedness of costs.

Control spaces need not be compact.

Abstract

For time-inconsistent stochastic controls in discrete time and finite horizon, an open problem in Bj\"ork and Murgoci (Finance Stoch, 2014) is the existence of an equilibrium control. A nonrandomized Borel measurable Markov equilibrium policy exists if the objective is inf-compact in every time step. We provide a sufficient condition for the inf-compactness and thus existence, with costs that are lower semicontinuous (l.s.c.) and bounded from below and transition kernels that are continuous in controls under given states. The control spaces need not to be compact.