Policy gradient learning methods for stochastic control with exit time and applications to share repurchase pricing

TL;DR

This paper introduces policy gradient algorithms for stochastic control problems with exit times, demonstrating their effectiveness in share repurchase pricing and outperforming traditional PDE and neural network methods.

Contribution

It presents novel policy gradient methods for exit time stochastic control, incorporating randomized policies and applying them to financial share repurchase problems.

Findings

Outperform PDE and neural network methods in share repurchase pricing

Effective handling of exit time issues via randomized policies

Flexible algorithms incorporating market conditions like transaction costs

Abstract

We develop policy gradients methods for stochastic control with exit time in a model-free setting. We propose two types of algorithms for learning either directly the optimal policy or by learning alternately the value function (critic) and the optimal control (actor). The use of randomized policies is crucial for overcoming notably the issue related to the exit time in the gradient computation. We demonstrate the effectiveness of our approach by implementing our numerical schemes in the application to the problem of share repurchase pricing. Our results show that the proposed policy gradient methods outperform PDE or other neural networks techniques in a model-based setting. Furthermore, our algorithms are flexible enough to incorporate realistic market conditions like e.g. price impact or transaction costs.