Continual Neural Mapping: Learning An Implicit Scene Representation from Sequential Observations

TL;DR

This paper introduces Continual Neural Mapping, a method enabling neural networks to learn and update implicit scene representations from sequential data without forgetting previous information, with applications in robotics and vision.

Contribution

It presents a novel continual learning framework for implicit neural scene representations, bridging batch training and streaming data scenarios.

Findings

Single network can continually represent scene geometry over time

Experience replay helps prevent catastrophic forgetting

Achieves a balance between accuracy and computational efficiency

Abstract

Recent advances have enabled a single neural network to serve as an implicit scene representation, establishing the mapping function between spatial coordinates and scene properties. In this paper, we make a further step towards continual learning of the implicit scene representation directly from sequential observations, namely Continual Neural Mapping. The proposed problem setting bridges the gap between batch-trained implicit neural representations and commonly used streaming data in robotics and vision communities. We introduce an experience replay approach to tackle an exemplary task of continual neural mapping: approximating a continuous signed distance function (SDF) from sequential depth images as a scene geometry representation. We show for the first time that a single network can represent scene geometry over time continually without catastrophic forgetting, while achieving promising trade-offs between accuracy and efficiency.