Model Confidence Bounds for Variable Selection

TL;DR

This paper introduces model confidence bounds (MCB) for variable selection, providing a group of nested models with confidence levels to assess uncertainty and visualize variability using the novel model uncertainty curve (MUC).

Contribution

The paper proposes the MCB methodology with a bootstrap algorithm and introduces the MUC for visualizing model selection uncertainty, advancing variable selection analysis.

Findings

MCB accurately captures model selection uncertainty.

The bootstrap algorithm achieves correct asymptotic coverage.

Real data examples demonstrate the method's effectiveness.

Abstract

In this article, we introduce the concept of model confidence bounds (MCB) for variable selection in the context of nested models. Similarly to the endpoints in the familiar confidence interval for parameter estimation, the MCB identifies two nested models (upper and lower confidence bound models) containing the true model at a given level of confidence. Instead of trusting a single selected model obtained from a given model selection method, the MCB proposes a group of nested models as candidates and the MCB's width and composition enable the practitioner to assess the overall model selection uncertainty. A new graphical tool --- the model uncertainty curve (MUC) --- is introduced to visualize the variability of model selection and to compare different model selection procedures. The MCB methodology is implemented by a fast bootstrap algorithm that is shown to yield the correct asymptotic coverage under rather general conditions. Our Monte Carlo simulations and real data examples confirm the validity and illustrate the advantages of the proposed method.