An efficient numerical algorithm for solving data driven feedback control problems

TL;DR

This paper introduces an efficient numerical algorithm for solving data-driven stochastic optimal control problems with partial observations, leveraging a stochastic optimization approach to estimate optimal feedback controls from observational data.

Contribution

The paper develops a novel computational framework and an efficient stochastic optimization algorithm for data-driven feedback control in stochastic systems with partial observations.

Findings

Effective estimation of optimal control as a conditional expectation.

Fast and feasible algorithm for real-time feedback control.

Applicable to systems governed by stochastic differential equations.

Abstract

The goal of this paper is to solve a class of stochastic optimal control problems numerically, in which the state process is governed by an It\^o type stochastic differential equation with control process entering both in the drift and the diffusion, and is observed partially. The optimal control of feedback form is determined based on the available observational data. We call this type of control problems the data driven feedback control. The computational framework that we introduce to solve such type of problems aims to find the best estimate for the optimal control as a conditional expectation given the observational information. To make our method feasible in providing timely feedback to the controlled system from data, we develop an efficient stochastic optimization algorithm to implement our computational framework.