A novel image tag completion method based on convolutional neural network

TL;DR

This paper introduces a new CNN-based method for completing incomplete image tags, jointly learning CNN parameters, linear predictors, and tags to improve image annotation accuracy.

Contribution

It proposes a novel joint learning framework that estimates complete image tags from CNN features, addressing tag incompleteness in image annotation tasks.

Findings

Demonstrates effectiveness on benchmark datasets

Reduces estimation error and model complexity

Improves image tag completion accuracy

Abstract

In the problems of image retrieval and annotation, complete textual tag lists of images play critical roles. However, in real-world applications, the image tags are usually incomplete, thus it is important to learn the complete tags for images. In this paper, we study the problem of image tag complete and proposed a novel method for this problem based on a popular image representation method, convolutional neural network (CNN). The method estimates the complete tags from the convolutional filtering outputs of images based on a linear predictor. The CNN parameters, linear predictor, and the complete tags are learned jointly by our method. We build a minimization problem to encourage the consistency between the complete tags and the available incomplete tags, reduce the estimation error, and reduce the model complexity. An iterative algorithm is developed to solve the minimization problem. Experiments over benchmark image data sets show its effectiveness.