Learn-to-Score: Efficient 3D Scene Exploration by Predicting View Utility

TL;DR

This paper introduces a learned utility function using a 3D CNN to guide drone exploration of unknown scenes, improving efficiency and robustness over traditional handcrafted methods.

Contribution

We develop a novel volumetric scene representation and train a 3D CNN to predict view usefulness, enabling real-time exploration of unknown environments.

Findings

Outperforms traditional utility functions in reconstruction quality

Robust to sensor noise in simulated experiments

Effective on large urban scene models

Abstract

Camera equipped drones are nowadays being used to explore large scenes and reconstruct detailed 3D maps. When free space in the scene is approximately known, an offline planner can generate optimal plans to efficiently explore the scene. However, for exploring unknown scenes, the planner must predict and maximize usefulness of where to go on the fly. Traditionally, this has been achieved using handcrafted utility functions. We propose to learn a better utility function that predicts the usefulness of future viewpoints. Our learned utility function is based on a 3D convolutional neural network. This network takes as input a novel volumetric scene representation that implicitly captures previously visited viewpoints and generalizes to new scenes. We evaluate our method on several large 3D models of urban scenes using simulated depth cameras. We show that our method outperforms existing utility measures in terms of reconstruction performance and is robust to sensor noise.