DiffVAS: Diffusion-Guided Visual Active Search in Partially Observable Environments

TL;DR

DiffVAS introduces a diffusion-guided, target-conditioned policy for visual active search in partially observable environments, enabling simultaneous multi-object search and outperforming existing methods.

Contribution

It proposes a novel diffusion-based reconstruction approach combined with reinforcement learning for effective multi-object search in realistic, limited-visibility scenarios.

Findings

DiffVAS significantly outperforms state-of-the-art methods on multiple datasets.

The diffusion model enables accurate reconstruction of partially observed geospatial areas.

The approach allows for efficient multi-target search in real-world applications.

Abstract

Visual active search (VAS) has been introduced as a modeling framework that leverages visual cues to direct aerial (e.g., UAV-based) exploration and pinpoint areas of interest within extensive geospatial regions. Potential applications of VAS include detecting hotspots for rare wildlife poaching, aiding search-and-rescue missions, and uncovering illegal trafficking of weapons, among other uses. Previous VAS approaches assume that the entire search space is known upfront, which is often unrealistic due to constraints such as a restricted field of view and high acquisition costs, and they typically learn policies tailored to specific target objects, which limits their ability to search for multiple target categories simultaneously. In this work, we propose DiffVAS, a target-conditioned policy that searches for diverse objects simultaneously according to task requirements in partially observable environments, which advances the deployment of visual active search policies in real-world applications. DiffVAS leverages a diffusion model to reconstruct the entire geospatial area from sequentially observed partial glimpses, which enables a target-conditioned reinforcement learning-based planning module to effectively reason and guide subsequent search steps. Extensive experiments demonstrate that DiffVAS excels in searching diverse objects in partially observable environments, significantly surpassing state-of-the-art methods on several datasets.