Inverse stochastic optimal controls

TL;DR

This paper addresses an inverse problem in stochastic optimal control, establishing well-posedness using a stochastic maximum principle, and proposes a numerical method that effectively recovers unknown parameters even in high-dimensional cases.

Contribution

It introduces a well-posedness result for the inverse stochastic control problem and develops a numerical approach applicable to multi-dimensional systems.

Findings

The inverse problem is well-posed under mild conditions.

The proposed numerical method accurately recovers unknown parameters.

The method is effective even without explicit value function forms.

Abstract

We study an inverse problem of the stochastic optimal control of general diffusions with performance index having the quadratic penalty term of the control process. Under mild conditions on the system dynamics, the cost functions, and the optimal control process, we show that our inverse problem is well-posed using a stochastic maximum principle. Then, with the well-posedness, we reduce the inverse problem to some root finding problem of the expectation of a random variable involved with the value function, which has a unique solution. Based on this result, we propose a numerical method for our inverse problem by replacing the expectation above with arithmetic mean of observed optimal control processes and the corresponding state processes. The recent progress of numerical analyses of Hamilton-Jacobi-Bellman equations enables the proposed method to be implementable for multi-dimensional cases. In particular, with the help of the kernel-based collocation method for Hamilton-Jacobi-Bellman equations, our method for the inverse problems still works well even when an explicit form of the value function is unavailable. Several numerical experiments show that the numerical method recovers the unknown penalty parameter with high accuracy.