Minimum information Markov model

TL;DR

This paper introduces the Minimum Information Markov Model, a new statistical framework for analyzing high-dimensional time series data that leverages orthogonal parametrization and divergence minimization.

Contribution

It proposes a novel Markov model with orthogonal dependence and stationary distribution, including estimators and theoretical results for Gaussian cases.

Findings

Establishes the existence of optimal solutions for Gaussian autoregressive models.

Develops conditional likelihood and pseudo likelihood estimators.

Demonstrates effectiveness through simulations and real data applications.

Abstract

The analysis of high-dimensional time series data has become increasingly important across a wide range of fields. Recently, a method for constructing the minimum information Markov kernel on finite state spaces was established. In this study, we propose a statistical model based on a parametrization of its dependence function, which we call the \textit{Minimum Information Markov Model}. We show that its parametrization induces an orthogonal structure between the stationary distribution and the dependence function, and that the model arises as the optimal solution to a divergence rate minimization problem. In particular, for the Gaussian autoregressive case, we establish the existence of the optimal solution to this minimization problem, a nontrivial result requiring a rigorous proof. For parameter estimation, our approach exploits the conditional independence structure inherent in the model, which is supported by the orthogonality. Specifically, we develop several estimators, including conditional likelihood and pseudo likelihood estimators, for the minimum information Markov model in both univariate and multivariate settings. We demonstrate their practical performance through simulation studies and applications to real-world time series data.