Model Selection Principles in Misspecified Models

TL;DR

This paper introduces semi-Bayesian principles and a new information criterion for model selection that effectively handle misspecified models, outperforming traditional criteria in various scenarios.

Contribution

It proposes a novel semi-Bayesian framework and derives the semi-Bayesian information criterion (SIC) tailored for misspecified models, combining strengths of classical principles.

Findings

SIC decomposes into likelihood, model complexity, and misspecification penalties.

Numerical studies show SIC outperforms traditional criteria in misspecified settings.

SIC performs well in both correctly specified and misspecified models.

Abstract

Model selection is of fundamental importance to high dimensional modeling featured in many contemporary applications. Classical principles of model selection include the Kullback-Leibler divergence principle and the Bayesian principle, which lead to the Akaike information criterion and Bayesian information criterion when models are correctly specified. Yet model misspecification is unavoidable when we have no knowledge of the true model or when we have the correct family of distributions but miss some true predictor. In this paper, we propose a family of semi-Bayesian principles for model selection in misspecified models, which combine the strengths of the two well-known principles. We derive asymptotic expansions of the semi-Bayesian principles in misspecified generalized linear models, which give the new semi-Bayesian information criteria (SIC). A specific form of SIC admits a natural decomposition into the negative maximum quasi-log-likelihood, a penalty on model dimensionality, and a penalty on model misspecification directly. Numerical studies demonstrate the advantage of the newly proposed SIC methodology for model selection in both correctly specified and misspecified models.