Annotating Ambiguous Images: General Annotation Strategy for High-Quality Data with Real-World Biomedical Validation

TL;DR

This paper presents a new annotation strategy for ambiguous biomedical images, validated through extensive real-world testing, to improve label quality in challenging datasets.

Contribution

It introduces a structured flowchart-based approach for generating high-quality labels in ambiguous datasets, validated with real-world biomedical data.

Findings

Effective label quality improvement demonstrated

Validated with over 250,000 annotations

Outperforms existing strategies in biomedical imaging

Abstract

In the field of image classification, existing methods often struggle with biased or ambiguous data, a prevalent issue in real-world scenarios. Current strategies, including semi-supervised learning and class blending, offer partial solutions but lack a definitive resolution. Addressing this gap, our paper introduces a novel strategy for generating high-quality labels in challenging datasets. Central to our approach is a clearly designed flowchart, based on a broad literature review, which enables the creation of reliable labels. We validate our methodology through a rigorous real-world test case in the biomedical field, specifically in deducing height reduction from vertebral imaging. Our empirical study, leveraging over 250,000 annotations, demonstrates the effectiveness of our strategies decisions compared to their alternatives.