Composed Image Retrieval for Remote Sensing

TL;DR

This paper introduces composed image retrieval for remote sensing, combining image and textual queries to enhance search capabilities, and demonstrates a novel fusion method that leverages existing vision-language models without additional training.

Contribution

It presents a new composed image retrieval method for remote sensing that fuses image-to-image and text-to-image similarity, along with a new benchmark for evaluation.

Findings

Sets new state-of-the-art performance on the proposed benchmark.

Shows vision-language models are sufficient without further training.

Provides a publicly available code implementation.

Abstract

This work introduces composed image retrieval to remote sensing. It allows to query a large image archive by image examples alternated by a textual description, enriching the descriptive power over unimodal queries, either visual or textual. Various attributes can be modified by the textual part, such as shape, color, or context. A novel method fusing image-to-image and text-to-image similarity is introduced. We demonstrate that a vision-language model possesses sufficient descriptive power and no further learning step or training data are necessary. We present a new evaluation benchmark focused on color, context, density, existence, quantity, and shape modifications. Our work not only sets the state-of-the-art for this task, but also serves as a foundational step in addressing a gap in the field of remote sensing image retrieval. Code at: https://github.com/billpsomas/rscir