Estimation of Markov Chain via Rank-Constrained Likelihood

TL;DR

This paper introduces a non-convex, rank-constrained likelihood estimator for low-rank Markov chains, providing theoretical bounds, a novel optimization algorithm, and demonstrating superior empirical performance.

Contribution

It presents a new non-convex estimator for low-rank Markov chains, along with a DC programming algorithm and theoretical guarantees, advancing state-of-the-art estimation methods.

Findings

The estimator achieves better empirical performance than existing methods.

Statistical upper bounds are established for divergence and risk.

The approach effectively reveals a compressed state space.

Abstract

This paper studies the estimation of low-rank Markov chains from empirical trajectories. We propose a non-convex estimator based on rank-constrained likelihood maximization. Statistical upper bounds are provided for the Kullback-Leiber divergence and the $\ell_2$ risk between the estimator and the true transition matrix. The estimator reveals a compressed state space of the Markov chain. We also develop a novel DC (difference of convex function) programming algorithm to tackle the rank-constrained non-smooth optimization problem. Convergence results are established. Experiments show that the proposed estimator achieves better empirical performance than other popular approaches.