Parameter learning: stochastic optimal control approach with reinforcement learning

TL;DR

This paper introduces a stochastic optimal control framework combined with reinforcement learning to estimate unknown parameters in stochastic differential equations, enabling effective parameter learning through optimal feedback control.

Contribution

It develops a novel control-based method for parameter estimation in stochastic differential equations using reinforcement learning principles.

Findings

Optimal density functions are Gaussian for the cases studied.

The method effectively estimates unknown parameters in drift and diffusion terms.

The approach provides a new way to perform empirical analysis of stochastic models.

Abstract

In this study, we develop a stochastic optimal control approach with reinforcement learning structure to learn the unknown parameters appeared in the drift and diffusion terms of the stochastic differential equation. By choosing an appropriate cost functional, based on a classical optimal feedback control, we translate the original optimal control problem to a new control problem which takes place the unknown parameter as control, and the related optimal control can be used to estimate the unknown parameter. We establish the mathematical framework of the dynamic equation for the exploratory state, which is consistent with the existing results. Furthermore, we consider the linear stochastic differential equation case where the drift or diffusion term with unknown parameter. Then, we investigate the general case where both the drift and diffusion terms contain unknown parameters. For the above cases, we show that the optimal density function satisfies a Gaussian distribution which can be used to estimate the unknown parameter. Based on the obtained estimation of the parameters, we can do empirical analysis for a given model.