Self-supervised TransUNet for Ultrasound regional segmentation of the distal radius in children

TL;DR

This paper explores self-supervised learning with Masked Autoencoder for improving ultrasound segmentation of children's distal radius, aiming to reduce the need for extensive labeled data in medical imaging.

Contribution

It demonstrates how modifying SSL-MAE components enhances segmentation performance and evaluates the effectiveness of SSL pretraining for TransUNet in medical image analysis.

Findings

Modified SSL-MAE improves downstream segmentation results.

SSL pretraining alone does not outperform training without SSL.

Changing embedding and loss functions enhances SSL-MAE effectiveness.

Abstract

Supervised deep learning offers great promise to automate analysis of medical images from segmentation to diagnosis. However, their performance highly relies on the quality and quantity of the data annotation. Meanwhile, curating large annotated datasets for medical images requires a high level of expertise, which is time-consuming and expensive. Recently, to quench the thirst for large data sets with high-quality annotation, self-supervised learning (SSL) methods using unlabeled domain-specific data, have attracted attention. Therefore, designing an SSL method that relies on minimal quantities of labeled data has far-reaching significance in medical images. This paper investigates the feasibility of deploying the Masked Autoencoder for SSL (SSL-MAE) of TransUNet, for segmenting bony regions from children's wrist ultrasound scans. We found that changing the embedding and loss function in SSL-MAE can produce better downstream results compared to the original SSL-MAE. In addition, we determined that only pretraining TransUNet embedding and encoder with SSL-MAE does not work as well as TransUNet without SSL-MAE pretraining on downstream segmentation tasks.