The Use of Quantitative Metrics and Machine Learning to Predict Radiologist Interpretations of MRI Image Quality and Artifacts

TL;DR

This study develops machine learning models using quantitative MRI quality metrics to predict radiologist assessments of image quality and artifacts, aiming to enhance real-time image evaluation in clinical settings.

Contribution

The paper introduces a decision tree ensemble model trained on combined radiologist evaluations and quantitative metrics, demonstrating improved prediction of MRI image quality and artifacts.

Findings

Model AUROC for image quality: 0.77

Model AUROC for imaging artifacts: 0.78

Generalized models outperform individual models

Abstract

A dataset of 3D-GRE and 3D-TSE brain 3T post contrast T1-weighted images as part of a quality improvement project were collected and shown to five neuro-radiologists who evaluated each sequence for both image quality and imaging artifacts. The same scans were processed using the MRQy tool for objective, quantitative image quality metrics. Using the combined radiologist and quantitative metrics dataset, a decision tree classifier with a bagging ensemble approach was trained to predict radiologist assessment using the quantitative metrics. A machine learning model was developed for the following three tasks: (1) determine the best model / performance for each MRI sequence and evaluation metric, (2) determine the best model / performance across all MRI sequences for each evaluation metric, and (3) determine the best general model / performance across all MRI sequences and evaluations. Model performance for imaging artifact was slightly higher than image quality, for example, the final generalized model AUROC for image quality was 0.77 (0.41 - 0.84, 95% CI) while imaging artifact was 0.78 (0.60 - 0.93, 95% CI). Further, it was noted that the generalized model performed slightly better than the individual models (AUROC 0.69 for 3D-GRE image quality, for example), indicating the value in comprehensive training data for these applications. These models could be deployed in the clinic as automatic checks for real-time image acquisition to prevent patient re-scanning requiring another appointment after retrospective radiologist analysis or improve reader confidence in the study. Further work needs to be done to validate the model described here on an external dataset. The results presented here suggest that MRQy could be used as a foundation for quantitative metrics as a surrogate for radiologist assessment.