A Duality Framework for Stochastic Optimal Control of Complex Systems

TL;DR

This paper introduces a duality framework for stochastic optimal control in complex systems, enabling the derivation of Pareto optimal policies that minimize long-term average costs in interactive subsystems.

Contribution

It develops a duality-based approach to find Pareto optimal control policies for complex systems with interacting subsystems, including existence conditions and geometric insights.

Findings

Duality framework for Pareto optimal control policies

Conditions for existence of optimal solutions

Geometric interpretation of control policies

Abstract

We address the problem of minimizing the long-run expected average cost of a complex system consisting of interactive subsystems. We formulate a multiobjective optimization problem of the one-stage expected costs of the subsystems and provide a duality framework to prove that the control policy yielding the Pareto optimal solution minimizes the average cost criterion of the system. We provide the conditions of existence and a geometric interpretation of the solution. For practical situations with constraints consistent to those studied here, our results imply that the Pareto control policy may be of value when we seek to derive online the optimal control policy in complex systems.