Exploring visual language models as a powerful tool in the diagnosis of Ewing Sarcoma

TL;DR

This paper investigates the use of vision-language models for diagnosing Ewing Sarcoma from histopathological images, demonstrating improved accuracy and efficiency over traditional pre-training methods.

Contribution

It introduces the application of vision-language supervision for ES diagnosis, showing significant accuracy gains and reduced computational costs compared to standard methods.

Findings

VLS improves diagnostic accuracy for ES.

VLS reduces model complexity and training costs.

In-domain VLS models outperform ImageNet pre-trained models.

Abstract

Ewing's sarcoma (ES), characterized by a high density of small round blue cells without structural organization, presents a significant health concern, particularly among adolescents aged 10 to 19. Artificial intelligence-based systems for automated analysis of histopathological images are promising to contribute to an accurate diagnosis of ES. In this context, this study explores the feature extraction ability of different pre-training strategies for distinguishing ES from other soft tissue or bone sarcomas with similar morphology in digitized tissue microarrays for the first time, as far as we know. Vision-language supervision (VLS) is compared to fully-supervised ImageNet pre-training within a multiple instance learning paradigm. Our findings indicate a substantial improvement in diagnostic accuracy with the adaption of VLS using an in-domain dataset. Notably, these models not only enhance the accuracy of predicted classes but also drastically reduce the number of trainable parameters and computational costs.