Optimal control under uncertainty and Bayesian parameters adjustments

TL;DR

This paper introduces a Bayesian framework for optimal impulse control under parameter uncertainty, deriving a novel dynamic programming equation that accounts for evolving priors and complex control sets.

Contribution

It develops a new approach to impulse control problems with Bayesian parameter updates, including a quasi-variational PDE characterization of the optimal policy.

Findings

Dynamic programming equation derived for Bayesian impulse control

Numerical methods proposed for solving the PDE

Framework accommodates evolving priors and complex control sets

Abstract

We propose a general framework for studying optimal impulse control problem in the presence of uncertainty on the parameters. Given a prior on the distribution of the unknown parameters, we explain how it should evolve according to the classical Bayesian rule after each impulse. Taking these progressive prior-adjustments into account, we characterize the optimal policy through a quasi-variational parabolic equation, which can be solved numerically. The derivation of the dynamic programming equation seems to be new in this context. The main difficulty lies in the nature of the set of controls which depends in a non trivial way on the initial data through the filtration itself.