An Improved Objective Evaluation Measure for Border Detection in Dermoscopy Images

TL;DR

This paper evaluates five recent dermoscopy border detection methods using an improved objective measure, revealing that differences among methods are smaller than previously estimated by traditional metrics.

Contribution

It introduces the use of the Normalized Probabilistic Rand Index for more accurate evaluation of border detection methods in dermoscopy images.

Findings

Four methods perform similarly under the new measure

Differences are smaller than those indicated by XOR measure

Evaluation highlights the importance of robust ground-truth comparison

Abstract

Background: Dermoscopy is one of the major imaging modalities used in the diagnosis of melanoma and other pigmented skin lesions. Due to the difficulty and subjectivity of human interpretation, dermoscopy image analysis has become an important research area. One of the most important steps in dermoscopy image analysis is the automated detection of lesion borders. Although numerous methods have been developed for the detection of lesion borders, very few studies were comprehensive in the evaluation of their results. Methods: In this paper, we evaluate five recent border detection methods on a set of 90 dermoscopy images using three sets of dermatologist-drawn borders as the ground-truth. In contrast to previous work, we utilize an objective measure, the Normalized Probabilistic Rand Index, which takes into account the variations in the ground-truth images. Conclusion: The results demonstrate that the differences between four of the evaluated border detection methods are in fact smaller than those predicted by the commonly used XOR measure.