# Cox reduction and confidence sets of models: a theoretical elucidation

## TL;DR

This paper provides a theoretical analysis of model confidence sets in high-dimensional sparse regression, evaluating reduction schemes and improving the Cox and Battey method for better stability and understanding.

## Contribution

It offers a geometric characterization of models in confidence sets and evaluates reduction strategies, enhancing the original Cox and Battey approach.

## Key findings

- Characterizes models likely retained in confidence sets
- Evaluates penalized regression and marginal screening reduction schemes
- Identifies data features affecting reduction efficacy

## Abstract

For sparse high-dimensional regression problems, Cox and Battey [1, 9] emphasised the need for confidence sets of models: an enumeration of those small sets of variables that fit the data equivalently well in a suitable statistical sense. This is to be contrasted with the single model returned by penalised regression procedures, effective for prediction but potentially misleading for subject-matter understanding. The proposed construction of such sets relied on preliminary reduction of the full set of variables, and while various possibilities could be considered for this, [9] proposed a succession of regression fits based on incomplete block designs. The purpose of the present paper is to provide insight on both aspects of that work. For an unspecified reduction strategy, we begin by characterising models that are likely to be retained in the model confidence set, emphasising geometric aspects. We then evaluate possible reduction schemes based on penalised regression or marginal screening, before theoretically elucidating the reduction of [9]. We identify features of the covariate matrix that may reduce its efficacy, and indicate improvements to the original proposal. An advantage of the approach is its ability to reveal its own stability or fragility for the data at hand.