On the minimal penalty for Markov order estimation

TL;DR

This paper demonstrates that under certain conditions, the penalized likelihood estimator for Markov order is strongly consistent with minimal penalties, using empirical process theory techniques.

Contribution

It introduces a new proof method that does not depend on explicit MLE expressions, applicable to broader settings.

Findings

Penalties as slow as log log n ensure consistency

Likelihood ratio satisfies a law of iterated logarithm

Method applicable beyond Markov models

Abstract

We show that large-scale typicality of Markov sample paths implies that the likelihood ratio statistic satisfies a law of iterated logarithm uniformly to the same scale. As a consequence, the penalized likelihood Markov order estimator is strongly consistent for penalties growing as slowly as log log n when an upper bound is imposed on the order which may grow as rapidly as log n. Our method of proof, using techniques from empirical process theory, does not rely on the explicit expression for the maximum likelihood estimator in the Markov case and could therefore be applicable in other settings.