A computational framework for two-dimensional random walks with restarts

TL;DR

This paper develops a new computational framework for analyzing two-dimensional random walks with restart events, enabling the calculation of steady-state distributions in complex Markov processes with infinite-dimensional matrices.

Contribution

It extends existing matrix-analytic methods to handle processes with restart events, introducing an enriched algebraic approach for infinite support corrections.

Findings

The framework successfully computes steady-state probabilities for complex random walks.

Numerical experiments confirm the approach's accuracy and applicability.

Conditions are provided to ensure solutions belong to the enriched algebra.

Abstract

The treatment of two-dimensional random walks in the quarter plane leads to Markov processes which involve semi-infinite matrices having Toeplitz or block Toeplitz structure plus a low-rank correction. Finding the steady state probability distribution of the process requires to perform operations involving these structured matrices. We propose an extension of the framework of [5] which allows to deal with more general situations such as processes involving restart events. This is motivated by the need for modeling processes that can incur in unexpected failures like computer system reboots. Algebraically, this gives rise to corrections with infinite support that cannot be treated using the tools currently available in the literature. We present a theoretical analysis of an enriched Banach algebra that, combined with appropriate algorithms, enables the numerical treatment of these problems. The results are applied to the solution of bidimensional Quasi-Birth-Death processes with infinitely many phases which model random walks in the quarter plane, relying on the matrix analytic approach. This methodology reduces the problem to solving a quadratic matrix equation with coefficients of infinite size. We provide conditions on the transition probabilities which ensure that the solution of interest of the matrix equation belongs to the enriched algebra. The reliability of our approach is confirmed by extensive numerical experimentation on some case studies.