Annotation Order Matters: Recurrent Image Annotator for Arbitrary Length Image Tagging

TL;DR

This paper introduces a Recurrent Image Annotator model that generates image tags as variable-length sequences, aligning more naturally with human annotation practices and improving flexibility over fixed top-k methods.

Contribution

The paper presents a novel sequence-based image annotation model that predicts variable-length tags and highlights the importance of tag order during training for better performance.

Findings

RIA outperforms traditional fixed top-k methods.

Tag order in training significantly impacts annotation accuracy.

Model serves as a high-quality baseline for arbitrary length image tagging.

Abstract

Automatic image annotation has been an important research topic in facilitating large scale image management and retrieval. Existing methods focus on learning image-tag correlation or correlation between tags to improve annotation accuracy. However, most of these methods evaluate their performance using top-k retrieval performance, where k is fixed. Although such setting gives convenience for comparing different methods, it is not the natural way that humans annotate images. The number of annotated tags should depend on image contents. Inspired by the recent progress in machine translation and image captioning, we propose a novel Recurrent Image Annotator (RIA) model that forms image annotation task as a sequence generation problem so that RIA can natively predict the proper length of tags according to image contents. We evaluate the proposed model on various image annotation datasets. In addition to comparing our model with existing methods using the conventional top-k evaluation measures, we also provide our model as a high quality baseline for the arbitrary length image tagging task. Moreover, the results of our experiments show that the order of tags in training phase has a great impact on the final annotation performance.