An Online Updating Approach for Testing the Proportional Hazards Assumption with Streams of Survival Data

TL;DR

This paper introduces an online updating method for testing the proportional hazards assumption in survival analysis, enabling efficient, real-time analysis of streaming data with reduced computational load.

Contribution

It presents a novel online updating test statistic for the PH assumption that is computationally efficient and effective for large, streaming survival datasets.

Findings

The online test maintains correct size under the null hypothesis.

It has high power to detect violations of the PH assumption.

The method successfully analyzes big data exceeding single computer capacity.

Abstract

The Cox model, which remains as the first choice in analyzing time-to-event data even for large datasets, relies on the proportional hazards (PH) assumption. When survival data arrive sequentially in chunks, a fast and minimally storage intensive approach to test the PH assumption is desirable. We propose an online updating approach that updates the standard test statistic as each new block of data becomes available, and greatly lightens the computational burden. Under the null hypothesis of PH, the proposed statistic is shown to have the same asymptotic distribution as the standard version computed on the entire data stream with the data blocks pooled into one dataset. In simulation studies, the test and its variant based on most recent data blocks maintain their sizes when the PH assumption holds and have substantial power to detect different violations of the PH assumption. We also show in simulation that our approach can be used successfully with "big data" that exceed a single computer's computational resources. The approach is illustrated with the survival analysis of patients with lymphoma cancer from the Surveillance, Epidemiology, and End Results Program. The proposed test promptly identified deviation from the PH assumption that was not captured by the test based on the entire data.