Crowdsourcing of Real-world Image Annotation via Visual Properties

TL;DR

This paper presents a crowdsourcing framework that uses visual property constraints and an interactive question-based approach to improve real-world image annotation, addressing semantic gaps and reducing subjectivity.

Contribution

It introduces a novel interactive crowdsourcing method combining knowledge representation, NLP, and computer vision to enhance annotation quality and consistency.

Findings

Effective reduction of annotator subjectivity.

Improved annotation accuracy demonstrated in experiments.

Framework guides annotators via visual property-based questions.

Abstract

Recent advances in data-centric artificial intelligence highlight inherent limitations in object recognition datasets. One of the primary issues stems from the semantic gap problem, which results in complex many-to-many mappings between visual data and linguistic descriptions. This bias adversely affects performance in computer vision tasks. This paper proposes an image annotation methodology that integrates knowledge representation, natural language processing, and computer vision techniques, aiming to reduce annotator subjectivity by applying visual property constraints. We introduce an interactive crowdsourcing framework that dynamically asks questions based on a predefined object category hierarchy and annotator feedback, guiding image annotation by visual properties. Experiments demonstrate the effectiveness of this methodology, and annotator feedback is discussed to optimize the crowdsourcing setup.