Towards the Efficient Inference by Incorporating Automated Computational Phenotypes under Covariate Shift

TL;DR

This paper introduces a semi-supervised learning approach that effectively incorporates automated computational phenotypes (ACPs) under covariate shift, improving inference efficiency while maintaining validity in real-world biomedical data analysis.

Contribution

It develops doubly robust, semiparametrically efficient estimators that leverage ACPs for both labeled and unlabeled data under covariate shift, with a focus on efficiency gains from unlabeled data.

Findings

Incorporating ACPs for unlabeled data enhances efficiency.

The proposed estimators are doubly robust and semiparametrically efficient.

Empirical validation confirms practical advantages of the method.

Abstract

Collecting gold-standard phenotype data via manual extraction is typically labor-intensive and slow, whereas automated computational phenotypes (ACPs) offer a systematic and much faster alternative. However, simply replacing the gold-standard with ACPs, without acknowledging their differences, could lead to biased results and misleading conclusions. Motivated by the complexity of incorporating ACPs while maintaining the validity of downstream analyses, in this paper, we consider a semi-supervised learning setting that consists of both labeled data (with gold-standard) and unlabeled data (without gold-standard), under the covariate shift framework. We develop doubly robust and semiparametrically efficient estimators that leverage ACPs for general target parameters in the unlabeled and combined populations. In addition, we carefully analyze the efficiency gains achieved by incorporating ACPs, comparing scenarios with and without their inclusion. Notably, we identify that ACPs for the unlabeled data, instead of for the labeled data, drive the enhanced efficiency gains. To validate our theoretical findings, we conduct comprehensive synthetic experiments and apply our method to multiple real-world datasets, confirming the practical advantages of our approach. \hfill{\texttt{Code}: \href{https://github.com/brucejunjin/ICML2025-ACPCS}{\faGithub}}