Investigating the limited performance of a deep-learning-based SPECT denoising approach: An observer-study-based characterization

TL;DR

This study investigates why deep-learning-based SPECT denoising methods often underperform in signal detection tasks, using an observer-study approach to analyze their sensitivity to different signal properties and highlighting the need for task-based evaluation.

Contribution

The paper introduces an observer-study-based characterization method to evaluate the performance of DL-based denoising on specific signal properties in SPECT imaging, revealing limitations and guiding improvements.

Findings

DL denoising did not improve signal detection performance across tested signals.

Performance varies with signal size and contrast, indicating sensitivity issues.

Observer-study approach offers a new way to evaluate denoising methods for specific tasks.

Abstract

Multiple objective assessment of image-quality-based studies have reported that several deep-learning-based denoising methods show limited performance on signal-detection tasks. Our goal was to investigate the reasons for this limited performance. To achieve this goal, we conducted a task-based characterization of a DL-based denoising approach for individual signal properties. We conducted this study in the context of evaluating a DL-based approach for denoising SPECT images. The training data consisted of signals of different sizes and shapes within a clustered-lumpy background, imaged with a 2D parallel-hole-collimator SPECT system. The projections were generated at normal and 20% low count level, both of which were reconstructed using an OSEM algorithm. A CNN-based denoiser was trained to process the low-count images. The performance of this CNN was characterized for five different signal sizes and four different SBR by designing each evaluation as an SKE/BKS signal-detection task. Performance on this task was evaluated using an anthropomorphic CHO. As in previous studies, we observed that the DL-based denoising method did not improve performance on signal-detection tasks. Evaluation using the idea of observer-study-based characterization demonstrated that the DL-based denoising approach did not improve performance on the signal-detection task for any of the signal types. Overall, these results provide new insights on the performance of the DL-based denoising approach as a function of signal size and contrast. More generally, the observer study-based characterization provides a mechanism to evaluate the sensitivity of the method to specific object properties and may be explored as analogous to characterizations such as modulation transfer function for linear systems. Finally, this work underscores the need for objective task-based evaluation of DL-based denoising approaches.