A Time Consistent Formulation of Risk Constrained Stochastic Optimal Control

TL;DR

This paper develops a time consistent formulation for risk constrained stochastic optimal control, ensuring rational decision-making over time by leveraging a dynamic risk measure and Bellman optimality, demonstrated through Haviv's counter-example.

Contribution

It introduces an analytical 'risk-to-go' approach that guarantees time consistency in risk constrained control problems, addressing a key challenge in risk-sensitive decision making.

Findings

Established an analytical 'risk-to-go' for time consistent policies

Demonstrated the approach on Haviv's counter-example

Validated the effectiveness of the method in time inconsistent planning

Abstract

Time-consistency is an essential requirement in risk sensitive optimal control problems to make rational decisions. An optimization problem is time consistent if its solution policy does not depend on the time sequence of solving the optimization problem. On the other hand, a dynamic risk measure is time consistent if a certain outcome is considered less risky in the future implies this outcome is also less risky at current stage. In this paper, we study time-consistency of risk constrained problem where the risk metric is time consistent. From the Bellman optimality condition in [1], we establish an analytical "risk-to-go" that results in a time consistent optimal policy. Finally we demonstrate the effectiveness of the analytical solution by solving Haviv's counter-example [2] in time inconsistent planning.