DermINO: Hybrid Pretraining for a Versatile Dermatology Foundation Model

TL;DR

DermINO is a versatile dermatology foundation model trained on a large, diverse dataset using a hybrid pretraining approach, significantly improving performance across various clinical tasks and demonstrating robustness and high diagnostic accuracy.

Contribution

Introduces DermNIO, a novel hybrid pretraining framework combining self-supervised, semi-supervised, and knowledge-guided learning for dermatology, enhancing generalization and versatility.

Findings

Outperforms state-of-the-art models on 20 datasets.

Achieves 95.79% diagnostic accuracy in a clinician study.

Excels in both high-level and low-level dermatological tasks.

Abstract

Skin diseases impose a substantial burden on global healthcare systems, driven by their high prevalence (affecting up to 70% of the population), complex diagnostic processes, and a critical shortage of dermatologists in resource-limited areas. While artificial intelligence(AI) tools have demonstrated promise in dermatological image analysis, current models face limitations-they often rely on large, manually labeled datasets and are built for narrow, specific tasks, making them less effective in real-world settings. To tackle these limitations, we present DermNIO, a versatile foundation model for dermatology. Trained on a curated dataset of 432,776 images from three sources (public repositories, web-sourced images, and proprietary collections), DermNIO incorporates a novel hybrid pretraining framework that augments the self-supervised learning paradigm through semi-supervised learning and knowledge-guided prototype initialization. This integrated method not only deepens the understanding of complex dermatological conditions, but also substantially enhances the generalization capability across various clinical tasks. Evaluated across 20 datasets, DermNIO consistently outperforms state-of-the-art models across a wide range of tasks. It excels in high-level clinical applications including malignancy classification, disease severity grading, multi-category diagnosis, and dermatological image caption, while also achieving state-of-the-art performance in low-level tasks such as skin lesion segmentation. Furthermore, DermNIO demonstrates strong robustness in privacy-preserving federated learning scenarios and across diverse skin types and sexes. In a blinded reader study with 23 dermatologists, DermNIO achieved 95.79% diagnostic accuracy (versus clinicians' 73.66%), and AI assistance improved clinician performance by 17.21%.