Adaptive Distributionally Robust Optimal Control with Bayesian Ambiguity Sets

TL;DR

This paper introduces an adaptive distributionally robust optimal control model that updates ambiguity sets via Bayesian learning, providing a less conservative, data-efficient approach with theoretical guarantees and practical effectiveness demonstrated through inventory control experiments.

Contribution

It develops a novel Bayesian-based adaptive DROC framework with a tractable reformulation, convergence guarantees, and a new algorithm for episodic data scenarios.

Findings

The model achieves robust performance with limited data.

The BOCP algorithm converges efficiently and reliably.

Numerical results show improved adaptivity and robustness.

Abstract

In stochastic optimal control (SOC), uncertainty may arise from incomplete knowledge of the true probability distribution of the underlying environment, which is known as Knightian or epistemic uncertainty. Distributionally robust optimal control (DROC) models are subsequently proposed to tackle this source of uncertainty. While such models are effective in some practical applications, most existing DROC models are offline and can be overly conservative when data are scarce. Moreover, they cannot be applied to the case when samples are generated episodically. Motivated by the Bayesian SOC framework recently proposed by Shapiro et al.~\cite{shapiro2025episodic}, we propose an adaptive DROC model in which the ambiguity set is updated via Bayesian learning from new data. Under some moderate conditions, we derive a tractable risk-averse reformulation, establish consistency of the optimal value function and optimal policy for an infinite-horizon SOC and establish a finite-sample posterior credibility guarantee for the policy value induced by the proposed episodic Bayesian DROC model. We also study the stability and statistical robustness of the proposed model with respect to sample perturbations that often arise in data-driven environments. To solve the episodic Bayesian DROC model, we propose a Bellman-operator cutting-plane (BOCP) algorithm that is computationally efficient and provably convergent. Numerical results on an inventory control problem demonstrate the effectiveness, adaptivity, and robust performance of the proposed model and algorithm.