Optimal control on finite graphs: asymptotic optimal controls and ergodic constant in the case of entropic costs

TL;DR

This paper studies optimal control problems on finite graphs with entropic costs, showing that the associated nonlinear differential equations can be linearized, enabling easy computation of asymptotic controls and ergodic constants using matrix analysis.

Contribution

It introduces a method to linearize nonlinear differential equations in finite graph control problems with entropic costs, simplifying the computation of key quantities.

Findings

Linearization of nonlinear differential equations via duality between entropy and exponential.

Explicit formulas for asymptotic optimal control and ergodic constant on connected graphs.

Application of classical matrix analysis tools for efficient computation.

Abstract

For optimal control problems on finite graphs in continuous time, the dynamic programming principle leads to value functions characterized by systems of nonlinear ordinary differential equations. In this paper, we consider the case of entropic costs for which the nonlinear differential equations can be transformed into linear ones thanks to a change of variables linked to the classical duality between entropy and exponential. When the graph is connected, we show that the asymptotic optimal control and the ergodic constant can be computed very easily with classical tools of matrix analysis.