Latent Diffusion Models with Image-Derived Annotations for Enhanced AI-Assisted Cancer Diagnosis in Histopathology

TL;DR

This paper introduces a method to generate synthetic histopathology images using latent diffusion models with image-derived prompts, improving data augmentation for cancer diagnosis AI systems.

Contribution

It presents a novel approach to construct structured prompts from image features for diffusion models, enhancing synthetic image quality in histopathology.

Findings

Improved FID score from 178.8 to 90.2 using image-derived prompts.

Pathologists struggle to distinguish synthetic from real images.

Synthetic data effectively trains AI models for cancer diagnosis.

Abstract

Artificial Intelligence (AI) based image analysis has an immense potential to support diagnostic histopathology, including cancer diagnostics. However, developing supervised AI methods requires large-scale annotated datasets. A potentially powerful solution is to augment training data with synthetic data. Latent diffusion models, which can generate high-quality, diverse synthetic images, are promising. However, the most common implementations rely on detailed textual descriptions, which are not generally available in this domain. This work proposes a method that constructs structured textual prompts from automatically extracted image features. We experiment with the PCam dataset, composed of tissue patches only loosely annotated as healthy or cancerous. We show that including image-derived features in the prompt, as opposed to only healthy and cancerous labels, improves the Fr\'echet Inception Distance (FID) from 178.8 to 90.2. We also show that pathologists find it challenging to detect synthetic images, with a median sensitivity/specificity of 0.55/0.55. Finally, we show that synthetic data effectively trains AI models.