Is Dataset Quality Still a Concern in Diagnosis Using Large Foundation Model?

TL;DR

This paper investigates the robustness of large foundation models in medical fundus diagnosis, finding they are more resilient to dataset quality issues than traditional models and that fine-tuning can further mitigate these effects.

Contribution

It demonstrates the robustness of large foundation models to dataset quality issues in medical diagnosis and shows fine-tuning as an effective method to improve their performance.

Findings

LFM is more resilient to image quality and dataset bias than CNNs.

Fine-tuning significantly improves LFM performance on low-quality datasets.

LFM outperforms traditional models in robustness to dataset issues.

Abstract

Recent advancements in pre-trained large foundation models (LFM) have yielded significant breakthroughs across various domains, including natural language processing and computer vision. These models have been particularly impactful in the domain of medical diagnostic tasks. With abundant unlabeled data, an LFM has been developed for fundus images using the Vision Transformer (VIT) and a self-supervised learning framework. This LFM has shown promising performance in fundus disease diagnosis across multiple datasets. On the other hand, deep learning models have long been challenged by dataset quality issues, such as image quality and dataset bias. To investigate the influence of data quality on LFM, we conducted explorations in two fundus diagnosis tasks using datasets of varying quality. Specifically, we explored the following questions: Is LFM more robust to image quality? Is LFM affected by dataset bias? Can fine-tuning techniques alleviate these effects? Our investigation found that LFM exhibits greater resilience to dataset quality issues, including image quality and dataset bias, compared to typical convolutional networks. Furthermore, we discovered that overall fine-tuning is an effective adapter for LFM to mitigate the impact of dataset quality issues.