Confidence intervals for the Cox model test error from cross-validation

TL;DR

This paper investigates the coverage properties of confidence intervals for test error in the Cox proportional hazards model, highlighting issues with standard CV intervals and proposing generalized nested CV methods.

Contribution

It extends the nested CV approach to the Cox model, providing new methods for more accurate confidence intervals for test error in survival analysis.

Findings

Standard CV confidence intervals often have below-nominal coverage.

Nested CV improves the accuracy of confidence intervals.

The paper explores different test error metrics for the Cox model.

Abstract

Cross-validation (CV) is one of the most widely used techniques in statistical learning for estimating the test error of a model, but its behavior is not yet fully understood. It has been shown that standard confidence intervals for test error using estimates from CV may have coverage below nominal levels. This phenomenon occurs because each sample is used in both the training and testing procedures during CV and as a result, the CV estimates of the errors become correlated. Without accounting for this correlation, the estimate of the variance is smaller than it should be. One way to mitigate this issue is by estimating the mean squared error of the prediction error instead using nested CV. This approach has been shown to achieve superior coverage compared to intervals derived from standard CV. In this work, we generalize the nested CV idea to the Cox proportional hazards model and explore various choices of test error for this setting.