Are Information criteria good enough to choose the right the number of regimes in Hidden Markov Models?

TL;DR

This paper evaluates the effectiveness of traditional information criteria versus new goodness-of-fit tests for selecting the number of regimes in Hidden Markov Models, considering various observation types.

Contribution

It introduces a new goodness-of-fit test for HMMs with covariates and compares its performance with classical criteria across different data types.

Findings

Goodness-of-fit tests perform well in finite samples.

Classical criteria may be less reliable for complex data.

New criterion shows promise for diverse observation distributions.

Abstract

Selecting the number of regimes in Hidden Markov models is an important problem. There are many criteria that are used to select this number, such as Akaike information criterion (AIC), Bayesian information criterion (BIC), integrated completed likelihood (ICL), deviance information criterion (DIC), and Watanabe-Akaike information criterion (WAIC), to name a few. In this article, we introduced goodness-of-fit tests for general Hidden Markov models with covariates, where the distribution of the observations is arbitrary, i.e., continuous, discrete, or a mixture of both. Then, a selection procedure is proposed based on this goodness-of-fit test. The main aim of this article is to compare the classical information criterion with the new criterion, when the outcome is either continuous, discrete or zero-inflated. Numerical experiments assess the finite sample performance of the goodness-of-fit tests, and comparisons between the different criteria are made.