A Review and Classification of Model Uncertainty

TL;DR

This paper reviews various definitions and classifications of model uncertainty in statistics and machine learning, emphasizing its importance for valid inference and proposing solutions to address it.

Contribution

It categorizes model uncertainty into three types, clarifies their interpretations, and discusses strategies to incorporate uncertainty into statistical inference.

Findings

Identifies three types of model uncertainty: true model uncertainty, selection uncertainty, and instability.

Highlights the consequences of ignoring model uncertainty in inference.

Provides practical solutions to account for model uncertainty in statistical analysis.

Abstract

Model uncertainty is a crucial issue in statistics, econometrics and machine learning, yet its definition remains ambiguous and is subject to various interpretations in the literature. So far, there has not been a universally accepted definition of model uncertainty. We review different understandings of model uncertainty and categorize them into three distinct types: uncertainty about the true model, model selection uncertainty, and model selection instability. We further offer interpretations and examples for a better illustration of these definitions. We also discuss the potential consequences of neglecting model uncertainty in the process of conducting statistical inference, and provide effective solutions to these problems. Our aim is to help researchers better understand the concept of model uncertainty and obtain valid statistical inference results on the premise of its existence.