On the Products of Stochastic and Diagonal Matrices

TL;DR

This paper introduces Tilted matrices, a new class formed by normalizing the product of a stochastic and diagonal matrix, and analyzes their properties and convergence in Markov Decision Processes.

Contribution

It defines Tilted matrices and provides new results on their products and convergence rates, advancing analysis techniques for Markov Decision Processes.

Findings

Tilted matrices are a new class with desirable properties.

Convergence rate results for products of Tilted reversible matrices.

Enhanced understanding of matrix products in Markov Decision Processes.

Abstract

Consider a stochastic matrix $P$ and diagonal matrix $D.$ In this work, we introduce Tilted matrices. A Tilted matrix is the product $D'PD$, where $D'$ is a diagonal normalization that makes the product stochastic. We then provide several results on products of Tilted matrices, which can be desirable for analyses of Markov Decision Processes. Lastly, we obtain a convergence rate result for the product of Tilted reversible matrices.