A semi-automatic method to guide the choice of ridge parameter in ridge regression

TL;DR

This paper introduces a semi-automatic method for selecting the ridge parameter in ridge regression, specifically tailored for high-dimensional genomic data, to improve out-of-sample prediction accuracy.

Contribution

The authors propose a variance-based ridge parameter selection method that enhances prediction performance in high-dimensional settings, extending to binary outcomes and real genomic data.

Findings

Outperforms subset selection and HyperLasso in simulated data

Improves prediction error in high-dimensional genomic data

Effective for binary case-control outcomes

Abstract

We consider the application of a popular penalised regression method, Ridge Regression, to data with very high dimensions and many more covariates than observations. Our motivation is the problem of out-of-sample prediction and the setting is high-density genotype data from a genome-wide association or resequencing study. Ridge regression has previously been shown to offer improved performance for prediction when compared with other penalised regression methods. One problem with ridge regression is the choice of an appropriate parameter for controlling the amount of shrinkage of the coefficient estimates. Here we propose a method for choosing the ridge parameter based on controlling the variance of the predicted observations in the model. Using simulated data, we demonstrate that our method outperforms subset selection based on univariate tests of association and another penalised regression method, HyperLasso regression, in terms of improved prediction error. We extend our approach to regression problems when the outcomes are binary (representing cases and controls, as is typically the setting for genome-wide association studies) and demonstrate the method on a real data example consisting of case-control and genotype data on Bipolar Disorder, taken from the Wellcome Trust Case Control Consortium and the Genetic Association Information Network.