Incremental Image Labeling via Iterative Refinement

TL;DR

This paper presents an iterative refinement approach for image labeling that leverages knowledge representation to improve data quality and semantic alignment, addressing the semantic gap in multimedia datasets.

Contribution

It introduces a KR-based iterative refinement method for organizing image labels hierarchically, enhancing semantic consistency in data annotation.

Findings

Preliminary results show improved labeling quality.

Method effectively aligns visual objects with linguistic descriptions.

Abstract

Data quality is critical for multimedia tasks, while various types of systematic flaws are found in image benchmark datasets, as discussed in recent work. In particular, the existence of the semantic gap problem leads to a many-to-many mapping between the information extracted from an image and its linguistic description. This unavoidable bias further leads to poor performance on current computer vision tasks. To address this issue, we introduce a Knowledge Representation (KR)-based methodology to provide guidelines driving the labeling process, thereby indirectly introducing intended semantics in ML models. Specifically, an iterative refinement-based annotation method is proposed to optimize data labeling by organizing objects in a classification hierarchy according to their visual properties, ensuring that they are aligned with their linguistic descriptions. Preliminary results verify the effectiveness of the proposed method.