# Prediction Weighted Maximum Frequency Selection

## TL;DR

This paper introduces a novel variable selection method combining adaptive LASSO and Elastic-Net estimators with a prediction-weighted maximum frequency approach, achieving consistent model selection and improved finite sample performance.

## Contribution

It develops a new strategy for variable selection that enhances finite sample performance and consistency using prediction-weighted maximum frequency models with adaptive estimators.

## Key findings

- Achieves consistent model selection with divergence of p_n.
- Demonstrates excellent finite sample performance in simulations.
- Extends methodology to generalized linear models.

## Abstract

Shrinkage estimators that possess the ability to produce sparse solutions have become increasingly important to the analysis of today's complex datasets. Examples include the LASSO, the Elastic-Net and their adaptive counterparts. Estimation of penalty parameters still presents difficulties however. While variable selection consistent procedures have been developed, their finite sample performance can often be less than satisfactory. We develop a new strategy for variable selection using the adaptive LASSO and adaptive Elastic-Net estimators with $p_n$ diverging. The basic idea first involves using the trace paths of their LARS solutions to bootstrap estimates of maximum frequency (MF) models conditioned on dimension. Conditioning on dimension effectively mitigates overfitting, however to deal with underfitting, these MFs are then prediction-weighted, and it is shown that not only can consistent model selection be achieved, but that attractive convergence rates can as well, leading to excellent finite sample performance. Detailed numerical studies are carried out on both simulated and real datasets. Extensions to the class of generalized linear models are also detailed.