Uncertainty-Driven Active Vision for Implicit Scene Reconstruction

TL;DR

This paper introduces an uncertainty-driven active vision method for implicit scene reconstruction that intelligently selects views to improve reconstruction quality, outperforming baselines and enhancing shape understanding.

Contribution

It presents a novel view selection strategy based on occupancy uncertainty, improving implicit scene reconstruction with fewer views and demonstrating effectiveness on multiple datasets.

Findings

Achieves state-of-the-art occupancy reconstruction quality.

View selection based on uncertainty outperforms baseline methods.

Gradient-based search further enhances shape understanding.

Abstract

Multi-view implicit scene reconstruction methods have become increasingly popular due to their ability to represent complex scene details. Recent efforts have been devoted to improving the representation of input information and to reducing the number of views required to obtain high quality reconstructions. Yet, perhaps surprisingly, the study of which views to select to maximally improve scene understanding remains largely unexplored. We propose an uncertainty-driven active vision approach for implicit scene reconstruction, which leverages occupancy uncertainty accumulated across the scene using volume rendering to select the next view to acquire. To this end, we develop an occupancy-based reconstruction method which accurately represents scenes using either 2D or 3D supervision. We evaluate our proposed approach on the ABC dataset and the in the wild CO3D dataset, and show that: (1) we are able to obtain high quality state-of-the-art occupancy reconstructions; (2) our perspective conditioned uncertainty definition is effective to drive improvements in next best view selection and outperforms strong baseline approaches; and (3) we can further improve shape understanding by performing a gradient-based search on the view selection candidates. Overall, our results highlight the importance of view selection for implicit scene reconstruction, making it a promising avenue to explore further.