# Optimal control on graphs: existence, uniqueness, and long-term behavior

## TL;DR

This paper develops a comprehensive framework for continuous-time Markov chain control on finite graphs, analyzing long-term behavior, ergodic equations, and establishing foundational results in a discrete state space setting.

## Contribution

It introduces new theoretical results on existence, uniqueness, and long-term behavior of optimal controls for Markov chains on finite graphs.

## Key findings

- Analysis of the long-term behavior of value functions
- Results on the ergodic Hamilton-Jacobi equation
- Framework applicable to finite graph control problems

## Abstract

The literature on continuous-time stochastic optimal control seldom deals with the case of discrete state spaces. In this paper, we provide a general framework for the optimal control of continuous-time Markov chains on finite graphs. In particular, we provide results on the long-term behavior of value functions and optimal controls, along with results on the associated ergodic Hamilton-Jacobi equation.