A Bayesian Approach for Task-Specific Next-Best-View Selection with Uncertain Geometry

TL;DR

This paper introduces a Bayesian framework for task-specific next-best-view selection in 3D reconstruction, optimizing camera views based on the intended downstream task to reduce relevant uncertainty efficiently.

Contribution

The authors propose a novel Bayesian decision-theoretic approach that focuses on task-specific uncertainty reduction in 3D reconstruction, improving efficiency over prior methods.

Findings

Achieves better task performance with fewer views compared to baselines.

Effectively reduces uncertainty in task-relevant regions.

Demonstrates versatility across multiple downstream tasks.

Abstract

We develop a framework for task-specific active next-best-view selection in 3D reconstruction from point clouds, by casting the problem in the language of Bayesian decision theory. Our framework works by (a) placing a prior distribution over the space of implicit surfaces, (b) using recently-developed stochastic surface reconstruction methods to calculate the resulting posterior distribution, then (c) using the posterior distribution to carefully reason about which view to scan next. This enables us to perform camera selection in a manner that is directly optimized for the intended use of the reconstructed data - meaning, we reduce uncertainty only in those regions that make a difference in the task at hand, as opposed to prior approaches that reduce it uniformly across space. We evaluate our method across three distinct downstream tasks: semantic classification, segmentation, and PDE-guided physics simulation. Experimental results demonstrate that our framework achieves superior task performance with fewer views compared to commonly used baselines and prior general uncertainty-reduction techniques.