Confidence Distributions for FIC scores

TL;DR

This paper introduces confidence distributions for FIC scores, enhancing model assessment by quantifying estimation uncertainty, and proposes new tools like quantile FIC and model averaging methods for improved model selection.

Contribution

It develops confidence distribution-based FIC plots and introduces the quantile FIC and model averaging techniques, advancing model evaluation and selection methods.

Findings

Confidence distribution-based FIC plots provide better uncertainty quantification.

Quantile FIC helps address biases in traditional FIC procedures.

Model averaging with FIC-based weights improves estimation accuracy.

Abstract

When using the Focused Information Criterion (FIC) for assessing and ranking candidate models with respect to how well they do for a given estimation task, it is customary to produce a so-called FIC plot. This plot has the different point estimates along the y-axis and the root-FIC scores on the x-axis, these being the estimated root-mean-square scores. In this paper we address the estimation uncertainty involved in each of the points of such a FIC plot. This needs careful assessment of each of the estimators from the candidate models, taking also modelling bias into account, along with the relative precision of the associated estimated mean squared error quantities. We use confidence distributions for these endeavours. This leads to fruitful CD-FIC plots, helping the statistician to judge to what extent the seemingly best models really are better than other models, etc. These efforts also lead to two further developments. The first is a new tool for model selection, which we call the quantile FIC, which helps overcome certain difficulties associated with the usual FIC procedures, related to somewhat arbitrary schemes for handling estimated squared biases. A particular case is the median-FIC. The second development is to form model averaged estimators with fruitful weights determined by the relative sizes of the median- and quantile-FIC scores. And Mrs. Jones is pregnant.