Next-Best View Policy for 3D Reconstruction

TL;DR

This paper introduces Scan-RL, a learning-based algorithm for optimizing the sequence of viewpoints in 3D house reconstruction, outperforming traditional methods by reducing steps and distance needed for complete models.

Contribution

We propose a novel reinforcement learning approach, Scan-RL, for Next-Best View selection in 3D reconstruction, and create the Houses3K dataset for training and evaluation.

Findings

Scan-RL reduces the number of steps needed for complete 3D scans.

A single NBV policy generalizes to unseen houses.

Scan-RL outperforms circular path baselines in efficiency.

Abstract

Manually selecting viewpoints or using commonly available flight planners like circular path for large-scale 3D reconstruction using drones often results in incomplete 3D models. Recent works have relied on hand-engineered heuristics such as information gain to select the Next-Best Views. In this work, we present a learning-based algorithm called Scan-RL to learn a Next-Best View (NBV) Policy. To train and evaluate the agent, we created Houses3K, a dataset of 3D house models. Our experiments show that using Scan-RL, the agent can scan houses with fewer number of steps and a shorter distance compared to our baseline circular path. Experimental results also demonstrate that a single NBV policy can be used to scan multiple houses including those that were not seen during training. The link to Scan-RL is available at https://github.com/darylperalta/ScanRL and Houses3K dataset can be found at https://github.com/darylperalta/Houses3K.