Identifying Sparse Low-Dimensional Structures in Markov Chains: A Nonnegative Matrix Factorization Approach

TL;DR

This paper introduces a nonnegative matrix factorization method to learn low-dimensional, sparse representations of large Markov chains, enabling efficient analysis of their structure and dynamics.

Contribution

It formulates the representation learning as a constrained NMF problem with a novel block coordinate gradient descent algorithm and analyzes its convergence.

Findings

Efficient algorithm for sparse low-dimensional Markov chain representations

Theoretical convergence guarantees for the proposed method

Potential for scalable analysis of large Markov models

Abstract

We consider the problem of learning low-dimensional representations for large-scale Markov chains. We formulate the task of representation learning as that of mapping the state space of the model to a low-dimensional state space, called the kernel space. The kernel space contains a set of meta states which are desired to be representative of only a small subset of original states. To promote this structural property, we constrain the number of nonzero entries of the mappings between the state space and the kernel space. By imposing the desired characteristics of the representation, we cast the problem as a constrained nonnegative matrix factorization. To compute the solution, we propose an efficient block coordinate gradient descent and theoretically analyze its convergence properties.