Bridging the Gap between Local Semantic Concepts and Bag of Visual Words for Natural Scene Image Retrieval

TL;DR

This paper explores how to improve natural scene image retrieval by combining local semantic concepts with the bag of visual words model to bridge the semantic gap in content-based image retrieval systems.

Contribution

It investigates the integration of semantic information with the bag of visual words model to enhance natural scene image retrieval performance.

Findings

Semantic information improves retrieval accuracy

Different semantic representation approaches are evaluated

Extensive experiments demonstrate effectiveness of the method

Abstract

This paper addresses the problem of semantic-based image retrieval of natural scenes. A typical content-based image retrieval system deals with the query image and images in the dataset as a collection of low-level features and retrieves a ranked list of images based on the similarities between features of the query image and features of images in the image dataset. However, top ranked images in the retrieved list, which have high similarities to the query image, may be different from the query image in terms of the semantic interpretation of the user which is known as the semantic gap. In order to reduce the semantic gap, this paper investigates how natural scene retrieval can be performed using the bag of visual word model and the distribution of local semantic concepts. The paper studies the efficiency of using different approaches for representing the semantic information, depicted in natural scene images, for image retrieval. An extensive experimental work has been conducted to study the efficiency of using semantic information as well as the bag of visual words model for natural and urban scene image retrieval.