Role of the Pretraining and the Adaptation data sizes for low-resource real-time MRI video segmentation

TL;DR

This study evaluates how pretraining and adaptation data sizes impact the performance of segmentation models in low-resource real-time MRI video analysis, emphasizing the importance of fine-tuning with limited data.

Contribution

It demonstrates that effective model adaptation for rtMRI segmentation can be achieved with as few as 15 frames, highlighting the importance of data size in low-resource settings.

Findings

Pretraining improves segmentation accuracy on unseen subjects and datasets.

Fine-tuning with minimal data (15 frames) significantly enhances model performance.

Models trained with limited data can approach matched condition accuracy.

Abstract

Real-time Magnetic Resonance Imaging (rtMRI) is frequently used in speech production studies as it provides a complete view of the vocal tract during articulation. This study investigates the effectiveness of rtMRI in analyzing vocal tract movements by employing the SegNet and UNet models for Air-Tissue Boundary (ATB)segmentation tasks. We conducted pretraining of a few base models using increasing numbers of subjects and videos, to assess performance on two datasets. First, consisting of unseen subjects with unseen videos from the same data source, achieving 0.33% and 0.91% (Pixel-wise Classification Accuracy (PCA) and Dice Coefficient respectively) better than its matched condition. Second, comprising unseen videos from a new data source, where we obtained an accuracy of 99.63% and 98.09% (PCA and Dice Coefficient respectively) of its matched condition performance. Here, matched condition performance refers to the performance of a model trained only on the test subjects which was set as a benchmark for the other models. Our findings highlight the significance of fine-tuning and adapting models with limited data. Notably, we demonstrated that effective model adaptation can be achieved with as few as 15 rtMRI frames from any new dataset.