Optimal Cox Regression Subsampling Procedure with Rare Events

TL;DR

This paper introduces an optimal subsampling method for Cox regression in large survival datasets with rare events, improving computational efficiency while maintaining statistical accuracy.

Contribution

It proposes a novel subsampling procedure that assigns optimal probabilities to censored data, enabling efficient Cox regression analysis on massive, rare-event survival datasets.

Findings

Estimator closely approximates full-data Cox regression results

Simulation studies demonstrate improved efficiency and accuracy

Applied method successfully to UK-biobank colorectal cancer data

Abstract

Massive sized survival datasets are becoming increasingly prevalent with the development of the healthcare industry. Such datasets pose computational challenges unprecedented in traditional survival analysis use-cases. A popular way for coping with massive datasets is downsampling them to a more manageable size, such that the computational resources can be afforded by the researcher. Cox proportional hazards regression has remained one of the most popular statistical models for the analysis of survival data to-date. This work addresses the settings of right censored and possibly left truncated data with rare events, such that the observed failure times constitute only a small portion of the overall sample. We propose Cox regression subsampling-based estimators that approximate their full-data partial-likelihood-based counterparts, by assigning optimal sampling probabilities to censored observations, and including all observed failures in the analysis. Asymptotic properties of the proposed estimators are established under suitable regularity conditions, and simulation studies are carried out to evaluate the finite sample performance of the estimators. We further apply our procedure on UK-biobank colorectal cancer genetic and environmental risk factors.