iMAP: Implicit Mapping and Positioning in Real-Time

TL;DR

iMAP demonstrates that a neural network can serve as a real-time, scene-specific implicit 3D map for handheld RGB-D SLAM, enabling efficient, detailed, and smooth scene reconstruction without prior data.

Contribution

This work introduces a novel real-time SLAM system using an MLP as the sole scene representation, trained live without prior data, and capable of dynamic scene modeling and tracking.

Findings

Achieves real-time tracking at 10 Hz and map updates at 2 Hz.

Provides efficient geometry representation with automatic detail control.

Enables smooth filling-in of unobserved regions like object backs.

Abstract

We show for the first time that a multilayer perceptron (MLP) can serve as the only scene representation in a real-time SLAM system for a handheld RGB-D camera. Our network is trained in live operation without prior data, building a dense, scene-specific implicit 3D model of occupancy and colour which is also immediately used for tracking. Achieving real-time SLAM via continual training of a neural network against a live image stream requires significant innovation. Our iMAP algorithm uses a keyframe structure and multi-processing computation flow, with dynamic information-guided pixel sampling for speed, with tracking at 10 Hz and global map updating at 2 Hz. The advantages of an implicit MLP over standard dense SLAM techniques include efficient geometry representation with automatic detail control and smooth, plausible filling-in of unobserved regions such as the back surfaces of objects.