ClinAlign: Scaling Healthcare Alignment from Clinician Preference

TL;DR

This paper introduces a two-stage framework using clinician-verified data and reusable principles to improve the alignment of large language models with medical standards, resulting in better performance on clinical tasks.

Contribution

It presents HealthRubrics and HealthPrinciples, novel datasets and methods for scalable, clinician-aligned supervision of LLMs in healthcare.

Findings

Model achieves 33.4% on HealthBench-Hard, outperforming larger models.

Framework enables scalable and resource-efficient clinical alignment.

Proposes tools for offline and inference-time model refinement.

Abstract

Although large language models (LLMs) demonstrate expert-level medical knowledge, aligning their open-ended outputs with fine-grained clinician preferences remains challenging. Existing methods often rely on coarse objectives or unreliable automated judges that are weakly grounded in professional guidelines. We propose a two-stage framework to address this gap. First, we introduce HealthRubrics, a dataset of 7,034 physician-verified preference examples in which clinicians refine LLM-drafted rubrics to meet rigorous medical standards. Second, we distill these rubrics into HealthPrinciples: 119 broadly reusable, clinically grounded principles organized by clinical dimensions, enabling scalable supervision beyond manual annotation. We use HealthPrinciples for (1) offline alignment by synthesizing rubrics for unlabeled queries and (2) an inference-time tool for guided self-revision. A 30B-A3B model trained with our framework achieves 33.4% on HealthBench-Hard, outperforming much larger models including Deepseek-R1 and o3, establishing a resource-efficient baseline for clinical alignment.