Nonparametric Adaptive Bayesian Stochastic Control Under Model Uncertainty

TL;DR

This paper introduces a nonparametric Bayesian approach using Dirichlet processes for stochastic control under model uncertainty, enabling online learning and avoiding model misspecification, with applications demonstrated in finance.

Contribution

It develops a novel nonparametric Bayesian methodology with numerical algorithms and Gaussian process surrogates for stochastic control under uncertainty, improving computational efficiency and robustness.

Findings

Outperforms parametric methods in financial applications

Enables online learning of unknown distributions

Reduces computational costs with surrogate models

Abstract

In this paper we propose a new methodology for solving a discrete time stochastic Markovian control problem under model uncertainty. By utilizing the Dirichlet process, we model the unknown distribution of the underlying stochastic process as a random probability measure and achieve online learning in a Bayesian manner. Our approach integrates optimizing and dynamic learning. When dealing with model uncertainty, the nonparametric framework allows us to avoid model misspecification that usually occurs in other classical control methods. Then, we develop a numerical algorithm to handle the infinitely dimensional state space in this setup and utilizes Gaussian process surrogates to obtain a functional representation of the value function in the Bellman recursion. We also build separate surrogates for optimal control to eliminate repeated optimizations on out-of-sample paths and bring computational speed-ups. Finally, we demonstrate the financial advantages of the nonparametric Bayesian framework compared to parametric approaches such as strong robust and time consistent adaptive.