Tuning Parameter Selection for Penalized Estimation via $R^2$

TL;DR

This paper introduces a correlation-based cross-validation method for tuning penalized estimators, improving support recovery and model interpretability over traditional error minimization strategies.

Contribution

It proposes a novel tuning parameter selection strategy based on maximizing correlation, applicable to all penalized least-squares estimators, enhancing support recovery and interpretability.

Findings

Effective support recovery demonstrated on functional variable selection.

Strategy closely related to Relaxed Lasso estimator.

Improved model interpretability over traditional methods.

Abstract

The tuning parameter selection strategy for penalized estimation is crucial to identify a model that is both interpretable and predictive. However, popular strategies (e.g., minimizing average squared prediction error via cross-validation) tend to select models with more predictors than necessary. This paper proposes a simple, yet powerful cross-validation strategy based on maximizing squared correlations between the observed and predicted values, rather than minimizing squared error loss for the purposes of support recovery. The strategy can be applied to all penalized least-squares estimators and we show that, under certain conditions, the metric implicitly performs a bias adjustment. Specific attention is given to the Lasso estimator, in which our strategy is closely related to the Relaxed Lasso estimator. We demonstrate our approach on a functional variable selection problem to identify optimal placement of surface electromyogram sensors to control a robotic hand prosthesis.