Assessing Impact of Data Quality in Early Post-Operative Glioblastoma Segmentation

TL;DR

This study examines how image and annotation quality affect deep learning models for early post-operative glioblastoma segmentation, finding that high-quality data improves performance but limits generalization to lower-quality data.

Contribution

The study quantifies the impact of data quality on model performance in early post-operative glioblastoma segmentation using a curated dataset with expert evaluations.

Findings

Models trained on high-quality images did not generalize well to low-quality images.

High-quality annotations achieved similar performance as the full dataset using only two-thirds of the data.

Both image and annotation quality significantly affect model performance in early post-operative segmentation.

Abstract

Quantification of the residual tumor from early post-operative magnetic resonance imaging (MRI) is essential in follow-up and treatment planning for glioblastoma patients. Residual tumor segmentation from early post-operative MRI is particularly challenging compared to the closely related task of pre-operative segmentation, as the tumor lesions are small, fragmented, and easily confounded with noise in the resection cavity. Recently, several studies successfully trained deep learning models for early post-operative segmentation, yet with subpar performances compared to the analogous task pre-operatively. In this study, the impact of image and annotation quality on model training and performance in early post-operative glioblastoma segmentation was assessed. A dataset consisting of early post-operative MRI scans from 423 patients and two hospitals in Norway and Sweden was assembled, for which image and annotation qualities were evaluated by expert neurosurgeons. The Attention U-Net architecture was trained with five-fold cross-validation on different quality-based subsets of the dataset in order to evaluate the impact of training data quality on model performance. Including low-quality images in the training set did not deteriorate performance on high-quality images. However, models trained on exclusively high-quality images did not generalize to low-quality images. Models trained on exclusively high-quality annotations reached the same performance level as the models trained on the entire dataset, using only two-thirds of the dataset. Both image and annotation quality had a significant impact on model performance. In dataset curation, images should ideally be representative of the quality variations in the real-world clinical scenario, and efforts should be made to ensure exact ground truth annotations of high quality.

Figures