A Zero-Shot Sketch-based Inter-Modal Object Retrieval Scheme for Remote Sensing Images

TL;DR

This paper introduces a novel zero-shot inter-modal retrieval method using sketches for remote sensing images, outperforming existing methods on a new dataset by effectively handling prototypical sketch representations.

Contribution

It presents a new inter-modal triplet-based zero-shot retrieval scheme for remote sensing data using sketches, along with a new dataset and comprehensive benchmarking.

Findings

The proposed method outperforms state-of-the-art zero-shot retrieval techniques.

The new Earth on Canvas dataset enables effective benchmarking of sketch-based remote sensing retrieval.

The scheme remains efficient even with marginally prototypical sketch representations.

Abstract

Conventional existing retrieval methods in remote sensing (RS) are often based on a uni-modal data retrieval framework. In this work, we propose a novel inter-modal triplet-based zero-shot retrieval scheme utilizing a sketch-based representation of RS data. The proposed scheme performs efficiently even when the sketch representations are marginally prototypical of the image. We conducted experiments on a new bi-modal image-sketch dataset called Earth on Canvas (EoC) conceived during this study. We perform a thorough bench-marking of this dataset and demonstrate that the proposed network outperforms other state-of-the-art methods for zero-shot sketch-based retrieval framework in remote sensing.