Automatic Labels are as Effective as Manual Labels in Biomedical Images Classification with Deep Learning

TL;DR

This study demonstrates that automatic labeling methods can produce labels with noise levels low enough (2-5%) to be as effective as manual labels for training deep learning models on biomedical image classification tasks, across multiple architectures and datasets.

Contribution

The paper provides evidence that automatic labels with minimal noise can replace manual labels in training deep learning models for biomedical image classification.

Findings

Automatic labels with 2-5% noise yield comparable performance to manual labels.

Models trained with automatic labels perform well across CNN and ViT architectures.

Automatic labeling is feasible for diverse biomedical datasets including binary, multiclass, and multilabel tasks.

Abstract

The increasing availability of biomedical data is helping to design more robust deep learning (DL) algorithms to analyze biomedical samples. Currently, one of the main limitations to train DL algorithms to perform a specific task is the need for medical experts to label data. Automatic methods to label data exist, however automatic labels can be noisy and it is not completely clear when automatic labels can be adopted to train DL models. This paper aims to investigate under which circumstances automatic labels can be adopted to train a DL model on the classification of Whole Slide Images (WSI). The analysis involves multiple architectures, such as Convolutional Neural Networks (CNN) and Vision Transformer (ViT), and over 10000 WSIs, collected from three use cases: celiac disease, lung cancer and colon cancer, which one including respectively binary, multiclass and multilabel data. The results allow identifying 10% as the percentage of noisy labels that lead to train competitive models for the classification of WSIs. Therefore, an algorithm generating automatic labels needs to fit this criterion to be adopted. The application of the Semantic Knowledge Extractor Tool (SKET) algorithm to generate automatic labels leads to performance comparable to the one obtained with manual labels, since it generates a percentage of noisy labels between 2-5%. Automatic labels are as effective as manual ones, reaching solid performance comparable to the one obtained training models with manual labels.