Weakly chained matrices, policy iteration, and impulse control

TL;DR

This paper investigates numerical methods for solving Hamilton-Jacobi-Bellman quasi-variational inequalities in stochastic and impulse control, introducing weakly chained matrices to analyze convergence and comparing different discretization schemes.

Contribution

It introduces the use of weakly chained diagonally dominant matrices for analyzing policy iteration convergence in HJBQVI problems and compares multiple discretization schemes.

Findings

Scheme (i) should be avoided due to convergence issues.

Weakly chained matrices provide a graph-theoretic framework for analysis.

Different schemes perform variably across control and pricing examples.

Abstract

This work is motivated by numerical solutions to Hamilton-Jacobi-Bellman quasi-variational inequalities (HJBQVIs) associated with combined stochastic and impulse control problems. In particular, we consider (i) direct control, (ii) penalized, and (iii) semi-Lagrangian discretization schemes applied to the HJBQVI problem. Scheme (i) takes the form of a Bellman problem involving an operator which is not necessarily contractive. We consider the well-posedness of the Bellman problem and give sufficient conditions for convergence of the corresponding policy iteration. To do so, we use weakly chained diagonally dominant matrices, which give a graph-theoretic characterization of weakly diagonally dominant M-matrices. We compare schemes (i)--(iii) under the following examples: (a) optimal control of the exchange rate, (b) optimal consumption with fixed and proportional transaction costs, and (c) pricing guaranteed minimum withdrawal benefits in variable annuities. We find that one should abstain from using scheme (i).