Clearing noisy annotations for computed tomography imaging

TL;DR

This paper introduces a three-stage annotation clearing algorithm for CT imaging segmentation that improves label quality by scoring annotators and nodules, and merging annotations based on confidence levels.

Contribution

It proposes a novel annotation clearing method with a scoring and merging process tailored for CT segmentation, enhancing label accuracy in multi-annotator scenarios.

Findings

Improves annotation quality in CT segmentation tasks.

Applicable to various tasks like classification and regression.

Enhances neural network training with cleaner labels.

Abstract

One of the problems on the way to successful implementation of neural networks is the quality of annotation. For instance, different annotators can annotate images in a different way and very often their decisions do not match exactly and in extreme cases are even mutually exclusive which results in noisy annotations and, consequently, inaccurate predictions. To avoid that problem in the task of computed tomography (CT) imaging segmentation we propose a clearing algorithm for annotations. It consists of 3 stages: - annotators scoring, which assigns a higher confidence level to better annotators; - nodules scoring, which assigns a higher confidence level to nodules confirmed by good annotators; - nodules merging, which aggregates annotations according to nodules confidence. In general, the algorithm can be applied to many different tasks (namely, binary and multi-class semantic segmentation, and also with trivial adjustments to classification and regression) where there are several annotators labeling each image.