Semantics Preserving Hierarchy based Retrieval of Indian heritage monuments

TL;DR

This paper introduces a hierarchy-based image retrieval framework that preserves semantic information for classifying and retrieving Indian heritage monuments, leveraging deep embeddings and a curated dataset.

Contribution

It proposes a novel hierarchy-preserving approach for monument classification and retrieval, utilizing deep embeddings and a re-ranking framework on a new Indian heritage dataset.

Findings

Effective classification at multiple hierarchy levels

Improved retrieval accuracy over baseline methods

Demonstrated on a large, curated Indian heritage dataset

Abstract

Monument classification can be performed on the basis of their appearance and shape from coarse to fine categories. Although there is much semantic information present in the monuments which is reflected in the eras they were built, its type or purpose, the dynasty which established it, etc. Particularly, Indian subcontinent exhibits a huge deal of variation in terms of architectural styles owing to its rich cultural heritage. In this paper, we propose a framework that utilizes hierarchy to preserve semantic information while performing image classification or image retrieval. We encode the learnt deep embeddings to construct a dictionary of images and then utilize a re-ranking framework on the the retrieved results using DeLF features. The semantic information preserved in these embeddings helps to classify unknown monuments at higher level of granularity in hierarchy. We have curated a large, novel Indian heritage monuments dataset comprising of images of historical, cultural and religious importance with subtypes of eras, dynasties and architectural styles. We demonstrate the performance of the proposed framework in image classification and retrieval tasks and compare it with other competing methods on this dataset.