CRAFT: Clinical Reward-Aligned Finetuning for Medical Image Synthesis

TL;DR

This paper introduces CRAFT, a reward-based finetuning framework for medical image synthesis that improves clinical alignment and reduces hallucinations in generated images across multiple modalities.

Contribution

CRAFT leverages clinical reward signals and multimodal models to enhance medical image synthesis, addressing limitations of traditional metrics and improving clinical relevance.

Findings

CRAFT improves the Clinical Alignment Score (CAS) across four modalities.

CRAFT reduces low-alignment tail by 5.5-34.7% points, averaging 20.4%.

CRAFT enhances downstream classification performance and reduces hallucinations.

Abstract

Foundation diffusion models can generate photorealistic natural images, but adapting them to medical imaging remains challenging. In medical adaptation, limited labeled data can exacerbate hallucination-like and clinically implausible synthesis, while existing metrics such as FID or Inception Score do not quantify per-image alignment with pathology-relevant criteria. We introduce the Clinical Alignment Score (CAS), a foundation-model-based proxy for clinical alignment that evaluates generated images along four complementary dimensions beyond visual fidelity. Building on CAS, we propose Clinical Reward-Aligned Finetuning (CRAFT), a reward-based adaptation framework that transfers medical knowledge from multimodal large language models and vision-language models through label-conditioned prompt enrichment, clinical checklists, and differentiable reward optimization. Across four diverse modalities, CRAFT improves CAS and downstream classification performance over strong adaptation baselines. Beyond average CAS gains, CRAFT reduces the empirical low-alignment tail below a real-image reference threshold by 5.5-34.7% points relative to the strongest baseline, corresponding to a 20.4% average relative reduction across datasets. These results indicate fewer hallucination-like generations under CAS, and are corroborated by out-of-family evaluator evaluation, structured checklist auditing, memorization analysis, and a blinded physician preference study on CheXpert.