Cross validating extensions of kernel, sparse or regular partial least squares regression models to censored data

TL;DR

This paper investigates the failure of standard cross-validation methods for extended partial least squares regression models in censored data, proposes new criteria, and demonstrates improved performance through extensive simulations.

Contribution

It reveals the inadequacy of traditional cross-validation for PLS extensions in Cox models and introduces new robust criteria that enhance model selection and performance.

Findings

Standard cross-validation fails for PLS extensions in Cox models.

New criteria improve component selection and model performance.

Proposed measures provide more robust evaluation in censored data contexts.

Abstract

When cross-validating standard or extended Cox models, the commonly used criterion is the cross-validated partial loglikelihood using a naive or a van Houwelingen scheme -to make efficient use of the death times of the left out data in relation to the death times of all the data-. Quite astonishingly, we will show, using a strong simulation study involving three different data simulation algorithms, that these two cross-validation methods fail with the extensions, either straightforward or more involved ones, of partial least squares regression to the Cox model. This is quite an interesting result for at least two reasons. Firstly, several nice features of PLS based models, including regularization, interpretability of the components, missing data support, data visualization thanks to biplots of individuals and variables -and even parsimony for SPLS based models-, account for a common use of these extensions by statisticians who usually select their hyperparameters using cross-validation. Secondly, they are almost always featured in benchmarking studies to assess the performance of a new estimation technique used in a high dimensional context and often show poor statistical properties. We carried out a vast simulation study to evaluate more than a dozen of potential cross-validation criteria, either AUC or prediction error based. Several of them lead to the selection of a reasonable number of components. Using these newly found cross-validation criteria to fit extensions of partial least squares regression to the Cox model, we performed a benchmark reanalysis that showed enhanced performances of these techniques. In addition, we defined a new robust measure based on the Schmid score and the R coefficient of determination for least absolute deviation: the integrated R Schmid Score weighted. Simulations were carried out using the R-package plsRcox.