A policy iteration algorithm for nonzero-sum stochastic impulse games

TL;DR

This paper introduces a new heuristic policy iteration algorithm for nonzero-sum stochastic impulse games, leveraging quasi-variational inequalities, and demonstrates its effectiveness through numerical tests in various scenarios.

Contribution

The paper presents the first numerical method for nonzero-sum stochastic impulse games based on a novel policy iteration approach.

Findings

Algorithm performs well in diverse situations

Successfully solves the only known analytically solvable example

Provides a practical computational tool for complex impulse games

Abstract

This work presents a novel policy iteration algorithm to tackle nonzero-sum stochastic impulse games arising naturally in many applications. Despite the obvious impact of solving such problems, there are no suitable numerical methods available, to the best of our knowledge. Our method relies on the recently introduced characterization of the value functions and Nash equilibrium via a system of quasi-variational inequalities. While our algorithm is heuristic and we do not provide a convergence analysis, numerical tests show that it performs convincingly in a wide range of situations, including the only analytically solvable example available in the literature at the time of writing.