ActiveRMAP: Radiance Field for Active Mapping And Planning

TL;DR

This paper introduces ActiveRMAP, an innovative framework that utilizes neural radiance fields for online active mapping and planning, enabling efficient 3D scene reconstruction and navigation with RGB images.

Contribution

It is the first to apply neural radiance fields to active vision tasks, integrating 3D reconstruction and planning in an online, iterative dual-stage optimization framework.

Findings

Achieves competitive results with offline methods.

Outperforms existing active NeRF-based reconstruction methods.

Demonstrates effective online 3D reconstruction and planning.

Abstract

A high-quality 3D reconstruction of a scene from a collection of 2D images can be achieved through offline/online mapping methods. In this paper, we explore active mapping from the perspective of implicit representations, which have recently produced compelling results in a variety of applications. One of the most popular implicit representations - Neural Radiance Field (NeRF), first demonstrated photorealistic rendering results using multi-layer perceptrons, with promising offline 3D reconstruction as a by-product of the radiance field. More recently, researchers also applied this implicit representation for online reconstruction and localization (i.e. implicit SLAM systems). However, the study on using implicit representation for active vision tasks is still very limited. In this paper, we are particularly interested in applying the neural radiance field for active mapping and planning problems, which are closely coupled tasks in an active system. We, for the first time, present an RGB-only active vision framework using radiance field representation for active 3D reconstruction and planning in an online manner. Specifically, we formulate this joint task as an iterative dual-stage optimization problem, where we alternatively optimize for the radiance field representation and path planning. Experimental results suggest that the proposed method achieves competitive results compared to other offline methods and outperforms active reconstruction methods using NeRFs.