Discriminative Representation Learning for Clinical Prediction

TL;DR

This paper introduces a supervised representation learning method for clinical prediction that outperforms traditional pretraining approaches by directly optimizing for outcome separation, improving model performance and efficiency.

Contribution

The authors propose a novel supervised framework that enhances clinical prediction by explicitly shaping representation geometry through outcome alignment, challenging the reliance on self-supervised pretraining.

Findings

Outperforms pretraining baselines in mortality and readmission tasks

Improves discrimination, calibration, and sample efficiency

Simplifies training to a single optimization stage

Abstract

Foundation models in healthcare have largely adopted self supervised pretraining objectives inherited from natural language processing and computer vision, emphasizing reconstruction and large scale representation learning prior to downstream adaptation. We revisit this paradigm in outcome centric clinical prediction settings and argue that, when high quality supervision is available, direct outcome alignment may provide a stronger inductive bias than generative pretraining. We propose a supervised deep learning framework that explicitly shapes representation geometry by maximizing inter class separation relative to within class variance, thereby concentrating model capacity along clinically meaningful axes. Across multiple longitudinal electronic health record tasks, including mortality and readmission prediction, our approach consistently outperforms masked, autoregressive, and contrastive pretraining baselines under matched model capacity. The proposed method improves discrimination, calibration, and sample efficiency, while simplifying the training pipeline to a single stage optimization. These findings suggest that in low entropy, outcome driven healthcare domains, supervision can act as the statistically optimal driver of representation learning, challenging the assumption that large scale self supervised pretraining is a prerequisite for strong clinical performance.