Model selection by minimum description length: Lower-bound sample sizes for the Fisher information approximation

TL;DR

This paper introduces a lower-bound sample size for the Fisher information approximation to improve model selection accuracy, addressing finite-sample issues by preventing misranking of model complexities.

Contribution

It proposes a novel lower-bound on sample size for FIA, ensuring reliable model selection in finite samples, which was previously problematic.

Findings

The lower-bound prevents FIA from misranking models in finite samples.

Application to multinomial processing tree models demonstrates effectiveness.

Provides practical guidelines for sample size in FIA-based model selection.

Abstract

The Fisher information approximation (FIA) is an implementation of the minimum description length principle for model selection. Unlike information criteria such as AIC or BIC, it has the advantage of taking the functional form of a model into account. Unfortunately, FIA can be misleading in finite samples, resulting in an inversion of the correct rank order of complexity terms for competing models in the worst case. As a remedy, we propose a lower-bound $N'$ for the sample size that suffices to preclude such errors. We illustrate the approach using three examples from the family of multinomial processing tree models.