Bag of Views: An Appearance-based Approach to Next-Best-View Planning for 3D Reconstruction

TL;DR

This paper introduces Bag-of-Views, an appearance-based model for efficient next-best-view planning in 3D reconstruction, reducing the number of views needed for high-quality models using reinforcement learning.

Contribution

The work presents a novel appearance-based approach for view planning and introduces VPT, a lightweight toolbox for training, testing, and dataset generation for 3D reconstruction tasks.

Findings

BoV reduces the number of views needed for high-quality reconstruction.

Reinforcement learning with BoV improves view planning efficiency.

VPT facilitates easy development and testing of view planning algorithms.

Abstract

UAV-based intelligent data acquisition for 3D reconstruction and monitoring of infrastructure has experienced an increasing surge of interest due to recent advancements in image processing and deep learning-based techniques. View planning is an essential part of this task that dictates the information capture strategy and heavily impacts the quality of the 3D model generated from the captured data. Recent methods have used prior knowledge or partial reconstruction of the target to accomplish view planning for active reconstruction; the former approach poses a challenge for complex or newly identified targets while the latter is computationally expensive. In this work, we present Bag-of-Views (BoV), a fully appearance-based model used to assign utility to the captured views for both offline dataset refinement and online next-best-view (NBV) planning applications targeting the task of 3D reconstruction. With this contribution, we also developed the View Planning Toolbox (VPT), a lightweight package for training and testing machine learning-based view planning frameworks, custom view dataset generation of arbitrary 3D scenes, and 3D reconstruction. Through experiments which pair a BoV-based reinforcement learning model with VPT, we demonstrate the efficacy of our model in reducing the number of required views for high-quality reconstructions in dataset refinement and NBV planning.