Distribution-Free Selection of Low-Risk Oncology Patients for Survival Beyond a Time Horizon

TL;DR

This paper introduces distribution-free methods for selecting low-risk oncology patients likely to survive beyond a certain time, using calibration and hypothesis testing frameworks with strong theoretical guarantees.

Contribution

It develops two novel distribution-free frameworks for patient screening, integrating calibration with Learn-Then-Test and FDR control via conformal p-values, applicable to censored survival data.

Findings

FDR-based screening is more powerful in practice.

LTT-based calibration provides stronger probabilistic guarantees.

Both methods are effectively applied to real oncology data.

Abstract

We study the problem of selecting a subset of patients who are unlikely to experience an adverse event within a fixed time horizon by calibrating a screening rule based on a black-box survival model. We consider two complementary, distribution-free frameworks for this task. The first extends classical calibration ideas -- estimating the event rate among selected patients using a hold-out dataset -- by integrating them with the Learn-Then-Test (LTT) framework, yielding high-probability guarantees for data-adaptively tuned screening rules. The second takes a different perspective by reformulating screening as a hypothesis testing problem on future patient outcomes, enabling false discovery rate (FDR) control via the Benjamini-Hochberg procedure applied to selective conformal p-values, and providing guarantees in expectation. We clarify the theoretical relationship between these approaches, explain how both can be adapted to right-censored time-to-event data via inverse probability of censoring weighting, and compare them empirically using simulations and oncology data from the Flatiron Health Research Database. Our results reveal a trade-off between efficiency and strength of guarantees: FDR-based screening is typically more powerful, while LTT-based calibration is more conservative but offers stronger guarantees. We also provide practical guidance on implementation and tuning.