Optimal control under unknown intensity with Bayesian learning

TL;DR

This paper develops a Bayesian framework for optimal control of systems driven by Poisson processes with unknown intensities, using dynamic programming and viscosity solutions to Hamilton-Jacobi-Bellman equations.

Contribution

It introduces a novel approach combining Bayesian learning with stochastic control under Poisson noise, providing a new characterization of the value function.

Findings

The value function is the unique viscosity solution to a finite-dimensional HJB equation.

The reformulation allows numerical solutions for control problems with unknown Poisson intensities.

The approach is applicable to neuroscience models involving stochastic neural dynamics.

Abstract

We investigate an optimal control problem motivated by neuroscience, where the dynamics is driven by a Poisson process with a controlled stochastic intensity and an unknown parameter. Given a prior distribution for the unknown parameter, we describe its evolution using Bayes' rule. We reformulate the optimization problem by applying Girsanov's theorem and establish a dynamic programming principle. Finally, we characterize the value function as the unique viscosity solution to a finite-dimensional Hamilton-Jacobi-Bellman equation, which can be solved numerically.