Scalable Reverse Image Search Engine for NASAWorldview

TL;DR

This paper presents a scalable, fast image similarity search engine for NASA's satellite imagery, reducing dataset exploration time from weeks to minutes using optimized CNN features and approximate nearest neighbor search.

Contribution

The authors developed a novel scalable image search system that compresses features and employs efficient search algorithms, enabling rapid retrieval from petabyte-scale satellite image datasets.

Findings

Search queries are completed in approximately 5 seconds on a single cloud VM.

Feature compression reduces image representation size from 2048 to 128 dimensions.

The system effectively handles petabyte-scale datasets with high speed and efficiency.

Abstract

Researchers often spend weeks sifting through decades of unlabeled satellite imagery(on NASA Worldview) in order to develop datasets on which they can start conducting research. We developed an interactive, scalable and fast image similarity search engine (which can take one or more images as the query image) that automatically sifts through the unlabeled dataset reducing dataset generation time from weeks to minutes. In this work, we describe key components of the end to end pipeline. Our similarity search system was created to be able to identify similar images from a potentially petabyte scale database that are similar to an input image, and for this we had to break down each query image into its features, which were generated by a classification layer stripped CNN trained in a supervised manner. To store and search these features efficiently, we had to make several scalability improvements. To improve the speed, reduce the storage, and shrink memory requirements for embedding search, we add a fully connected layer to our CNN make all images into a 128 length vector before entering the classification layers. This helped us compress the size of our image features from 2048 (for ResNet, which was initially tried as our featurizer) to 128 for our new custom model. Additionally, we utilize existing approximate nearest neighbor search libraries to significantly speed up embedding search. Our system currently searches over our entire database of images at 5 seconds per query on a single virtual machine in the cloud. In the future, we would like to incorporate a SimCLR based featurizing model which could be trained without any labelling by a human (since the classification aspect of the model is irrelevant to this use case).