RO-MAP: Real-Time Multi-Object Mapping with Neural Radiance Fields

TL;DR

RO-MAP introduces a real-time multi-object mapping system using neural radiance fields that localizes and reconstructs objects from monocular input without relying on 3D priors, suitable for SLAM and AR applications.

Contribution

It presents a novel pipeline combining neural radiance fields with lightweight SLAM for dynamic, multi-object mapping without pre-existing 3D models, operating in real-time.

Findings

Achieves real-time performance at 25Hz.

Produces semantic object maps with shape reconstruction.

Competitive with offline methods on synthetic and real datasets.

Abstract

Accurate perception of objects in the environment is important for improving the scene understanding capability of SLAM systems. In robotic and augmented reality applications, object maps with semantic and metric information show attractive advantages. In this paper, we present RO-MAP, a novel multi-object mapping pipeline that does not rely on 3D priors. Given only monocular input, we use neural radiance fields to represent objects and couple them with a lightweight object SLAM based on multi-view geometry, to simultaneously localize objects and implicitly learn their dense geometry. We create separate implicit models for each detected object and train them dynamically and in parallel as new observations are added. Experiments on synthetic and real-world datasets demonstrate that our method can generate semantic object map with shape reconstruction, and be competitive with offline methods while achieving real-time performance (25Hz). The code and dataset will be available at: https://github.com/XiaoHan-Git/RO-MAP