Skin-SOAP: A Weakly Supervised Framework for Generating Structured SOAP Notes

TL;DR

Skin-SOAP is a weakly supervised multimodal framework that generates structured SOAP notes from limited inputs, reducing manual annotation needs and clinician workload while maintaining high clinical relevance.

Contribution

It introduces a novel weakly supervised approach for automated SOAP note generation using multimodal data, with new metrics for evaluating clinical relevance.

Findings

Achieves performance comparable to GPT-4o, Claude, and DeepSeek Janus Pro.

Introduces MedConceptEval and CCS metrics for clinical relevance assessment.

Reduces reliance on manual annotations for clinical documentation.

Abstract

Skin carcinoma is the most prevalent form of cancer globally, accounting for over $8 billion in annual healthcare expenditures. Early diagnosis, accurate and timely treatment are critical to improving patient survival rates. In clinical settings, physicians document patient visits using detailed SOAP (Subjective, Objective, Assessment, and Plan) notes. However, manually generating these notes is labor-intensive and contributes to clinician burnout. In this work, we propose skin-SOAP, a weakly supervised multimodal framework to generate clinically structured SOAP notes from limited inputs, including lesion images and sparse clinical text. Our approach reduces reliance on manual annotations, enabling scalable, clinically grounded documentation while alleviating clinician burden and reducing the need for large annotated data. Our method achieves performance comparable to GPT-4o, Claude, and DeepSeek Janus Pro across key clinical relevance metrics. To evaluate this clinical relevance, we introduce two novel metrics MedConceptEval and Clinical Coherence Score (CCS) which assess semantic alignment with expert medical concepts and input features, respectively.