Addressing Visual Search in Open and Closed Set Settings

TL;DR

This paper introduces a novel, efficient method for visual search in large images that predicts objectness at low resolution to guide high-resolution detection, improving performance in both closed-set and open-set scenarios.

Contribution

It presents a new pixel-level objectness prediction technique from low-res images and a Bayesian-based open-set search strategy, reducing computational costs and enhancing detection accuracy.

Findings

Significantly outperforms baseline methods in detection tasks.

Reduces the number of high-resolution glimpses needed.

Effective in both closed-set and open-set visual search scenarios.

Abstract

Searching for small objects in large images is a task that is both challenging for current deep learning systems and important in numerous real-world applications, such as remote sensing and medical imaging. Thorough scanning of very large images is computationally expensive, particularly at resolutions sufficient to capture small objects. The smaller an object of interest, the more likely it is to be obscured by clutter or otherwise deemed insignificant. We examine these issues in the context of two complementary problems: closed-set object detection and open-set target search. First, we present a method for predicting pixel-level objectness from a low resolution gist image, which we then use to select regions for performing object detection locally at high resolution. This approach has the benefit of not being fixed to a predetermined grid, thereby requiring fewer costly high-resolution glimpses than existing methods. Second, we propose a novel strategy for open-set visual search that seeks to find all instances of a target class which may be previously unseen and is defined by a single image. We interpret both detection problems through a probabilistic, Bayesian lens, whereby the objectness maps produced by our method serve as priors in a maximum-a-posteriori approach to the detection step. We evaluate the end-to-end performance of both the combination of our patch selection strategy with this target search approach and the combination of our patch selection strategy with standard object detection methods. Both elements of our approach are seen to significantly outperform baseline strategies.