Duality and Policy Evaluation in Distributionally Robust Bayesian Diffusion Control

TL;DR

This paper introduces a distributionally robust Bayesian control framework for diffusion control problems under parameter uncertainty, addressing prior misspecification and distribution shifts with a duality-based optimization approach.

Contribution

It proposes a novel DRBC formulation with a duality result that simplifies prior evaluation and enables practical policy learning under uncertainty.

Findings

Efficient algorithm validated on linear-quadratic control example.

Demonstrated effectiveness in real-data portfolio selection.

Reduces over-pessimism compared to classical robust control.

Abstract

We study diffusion control problems under parameter uncertainty. Controllers based on plug-in estimation can be brittle due to potential distribution shifts. Bayesian control with a prior on the parameters offers a formulation with beliefs about such shifts. However, as with any Bayesian model, the prior may be misspecified. To mitigate misspecification and reduce over-pessimism compared to classical robust control approaches (e.g. \citet{hansen2008robustness}), we propose a distributionally robust Bayesian control (DRBC) formulation in which an adversary perturbs the prior within a divergence neighborhood of a baseline prior. We develop a strong duality result that reduces the distributionally robust prior evaluation to a low-dimensional optimization and yields a practical simulation-based policy evaluation and learning procedure with structured policy parameterizations. We validate the efficiency of the algorithm on a synthetic linear-quadratic control example and real-data portfolio selection.