DermoGPT: Open Weights and Open Data for Morphology-Grounded Dermatological Reasoning MLLMs

TL;DR

DermoGPT is a novel multimodal language model for dermatology that leverages open data, a comprehensive instruction corpus, and a new benchmark to improve diagnostic reasoning and clinical task performance.

Contribution

The paper introduces DermoInstruct, DermoBench, and DermoGPT, pioneering morphology-grounded dermatological reasoning with open data, tasks, and a specialized training framework.

Findings

DermoGPT outperforms 16 baselines across multiple clinical axes.

Significant reduction in human-AI diagnostic gap.

State-of-the-art results on dermatology reasoning tasks.

Abstract

Multimodal Large Language Models (MLLMs) show promise for medical applications, yet progress in dermatology lags due to limited training data, narrow task coverage, and lack of clinically-grounded supervision that mirrors expert diagnostic workflows. We present a comprehensive framework to address these gaps. First, we introduce DermoInstruct, a large-scale morphology-anchored instruction corpus comprising 211,243 images and 772,675 trajectories across five task formats, capturing the complete diagnostic pipeline from morphological observation and clinical reasoning to final diagnosis. Second, we establish DermoBench, a rigorous benchmark evaluating 11 tasks across four clinical axes: Morphology, Diagnosis, Reasoning, and Fairness, including a challenging subset of 3,600 expert-verified open-ended instances and human performance baselines. Third, we develop DermoGPT, a dermatology reasoning MLLM trained via supervised fine-tuning followed by our Morphologically-Anchored Visual-Inference-Consistent (MAVIC) reinforcement learning objective, which enforces consistency between visual observations and diagnostic conclusions. At inference, we deploy Confidence-Consistency Test-time adaptation (CCT) for robust predictions. Experiments show DermoGPT significantly outperforms 16 representative baselines across all axes, achieving state-of-the-art performance while substantially narrowing the human-AI gap. DermoInstruct, DermoBench and DermoGPT will be made publicly available at https://github.com/mendicant04/DermoGPT upon acceptance.