Prior Adaptive Semi-supervised Learning with Application to EHR Phenotyping

TL;DR

This paper introduces PASS, a semi-supervised learning method that leverages both labeled and unlabeled EHR data with surrogate variables to improve disease phenotyping accuracy, especially in high-dimensional settings.

Contribution

The paper proposes a novel prior adaptive semi-supervised estimator for EHR phenotyping that effectively utilizes surrogate variables and unlabeled data, with theoretical guarantees and practical validation.

Findings

PASS outperforms existing methods in simulations.

The method demonstrates improved phenotyping accuracy on rheumatoid arthritis data.

Asymptotic theory supports the estimator's consistency and efficiency.

Abstract

Electronic Health Records (EHR) data, a rich source for biomedical research, have been successfully used to gain novel insight into a wide range of diseases. Despite its potential, EHR is currently underutilized for discovery research due to it's major limitation in the lack of precise phenotype information. To overcome such difficulties, recent efforts have been devoted to developing supervised algorithms to accurately predict phenotypes based on relatively small training datasets with gold standard labels extracted via chart review. However, supervised methods typically require a sizable training set to yield generalizable algorithms especially when the number of candidate features, $p$, is large. In this paper, we propose a semi-supervised (SS) EHR phenotyping method that borrows information from both a small labeled data where both the label $Y$ and the feature set $X$ are observed and a much larger unlabeled data with observations on $X$ only as well as a surrogate variable $S$ that is predictive of $Y$ and available for all patients, under a high dimensional setting. Under a working prior assumption that $S$ is related to $X$ only through $Y$ and allowing it to hold approximately, we propose a prior adaptive semi-supervised (PASS) estimator that adaptively incorporates the prior knowledge by shrinking the estimator towards a direction derived under the prior. We derive asymptotic theory for the proposed estimator and demonstrate its superiority over existing estimators via simulation studies. The proposed method is applied to an EHR phenotyping study of rheumatoid arthritis at Partner's Healthcare.