Discrete-time Markov chains with random observation times

TL;DR

This paper introduces a new method for estimating transition matrices of Markov chains observed at random times, accommodating covariates, with proven convergence rates and effective simulation results.

Contribution

It presents a novel estimation approach using kernel methods for Markov chains with random observation times, including covariate dependence, and provides theoretical convergence guarantees.

Findings

Estimation performs well in simulations.

Kernel estimates have established uniform convergence rates.

Method works with as few as two observations per path.

Abstract

We propose a new approach for estimating the finite dimensional transition matrix of a Markov chain using a large number of independent sample paths observed at random times. The sample paths may be observed as few as two times, and the transitions are allowed to depend on covariates. Simple and easy to update kernel estimates are proposed, and their uniform convergence rates are derived. Simulation experiments show that our estimation approach performs well.