Segmentation with Multiple Acceptable Annotations: A Case Study of Myocardial Segmentation in Contrast Echocardiography

TL;DR

This paper introduces an extended Dice metric for evaluating myocardial segmentation in contrast echocardiography with multiple annotations, and incorporates it into a loss function to improve neural network training and performance.

Contribution

It proposes a novel extended Dice metric for performance evaluation and a new loss function that leverages multiple annotations in myocardial segmentation tasks.

Findings

The proposed loss function outperforms existing methods quantitatively.

Extended Dice better identifies segmentation results needing manual correction.

Neural networks trained with the new loss achieve more accurate segmentation.

Abstract

Most existing deep learning-based frameworks for image segmentation assume that a unique ground truth is known and can be used for performance evaluation. This is true for many applications, but not all. Myocardial segmentation of Myocardial Contrast Echocardiography (MCE), a critical task in automatic myocardial perfusion analysis, is an example. Due to the low resolution and serious artifacts in MCE data, annotations from different cardiologists can vary significantly, and it is hard to tell which one is the best. In this case, how can we find a good way to evaluate segmentation performance and how do we train the neural network? In this paper, we address the first problem by proposing a new extended Dice to effectively evaluate the segmentation performance when multiple accepted ground truth is available. Then based on our proposed metric, we solve the second problem by further incorporating the new metric into a loss function that enables neural networks to flexibly learn general features of myocardium. Experiment results on our clinical MCE data set demonstrate that the neural network trained with the proposed loss function outperforms those existing ones that try to obtain a unique ground truth from multiple annotations, both quantitatively and qualitatively. Finally, our grading study shows that using extended Dice as an evaluation metric can better identify segmentation results that need manual correction compared with using Dice.