Estimating the transition matrix of a Markov chain observed at random times

TL;DR

This paper introduces a new statistical method to estimate the transition matrix of a Markov chain observed at random, censored times, even when the time gaps and their distribution are unknown.

Contribution

It develops a consistent estimation technique for the transition kernel of a Markov chain with censored data and unknown time gaps, leveraging known unfeasible transitions.

Findings

The estimator is consistent and has a closed-form solution.

Asymptotic properties are established theoretically.

Numerical simulations demonstrate the estimator's effectiveness.

Abstract

In this paper we develop a statistical estimation technique to recover the transition kernel $P$ of a Markov chain $X=(X_m)_{m \in \mathbb N}$ in presence of censored data. We consider the situation where only a sub-sequence of $X$ is available and the time gaps between the observations are iid random variables. Under the assumption that neither the time gaps nor their distribution are known, we provide an estimation method which applies when some transitions in the initial Markov chain $X$ are known to be unfeasible. A consistent estimator of $P$ is derived in closed form as a solution of a minimization problem. The asymptotic performance of the estimator is then discussed in theory and through numerical simulations.