Exact test for Markov order

TL;DR

This paper introduces an exact, non-asymptotic test for determining the Markov order of a chain, utilizing a novel surrogate data algorithm to accurately assess the null hypothesis regardless of sample size.

Contribution

It presents a new exact testing procedure for Markov order that does not depend on asymptotic assumptions, using a unique surrogate data generation method.

Findings

The test is valid for any sample size.

The surrogate data algorithm guarantees uniform sampling.

The method accurately distinguishes between nth and (n+1)th order Markov chains.

Abstract

We describe an exact test of the null hypothesis that a Markov chain is nth order versus the alternate hypothesis that it is $(n+1)$-th order. The procedure does not rely on asymptotic properties, but instead builds up the test statistic distribution via surrogate data and is valid for any sample size. Surrogate data are generated using a novel algorithm that guarantees, per shot, a uniform sampling from the set of sequences that exactly match the nth order properties of the observed data.