TACO: Temporal Consensus Optimization for Continual Neural Mapping

TL;DR

TACO is a replay-free continual neural mapping framework that enforces temporal consensus with past models to adapt to dynamic environments efficiently.

Contribution

It introduces a novel temporal consensus optimization approach for continual neural mapping without replay, balancing memory efficiency and adaptability.

Findings

TACO outperforms existing continual learning methods in simulated and real-world experiments.

It effectively adapts to scene changes while maintaining memory and computation efficiency.

The approach enables reliable past geometry constraints without storing previous data.

Abstract

Neural implicit mapping has emerged as a powerful paradigm for robotic navigation and scene understanding. However, real-world robotic deployment requires continual adaptation to changing environments under strict memory and computation constraints, which existing mapping systems fail to support. Most prior methods rely on replaying historical observations to preserve consistency and assume static scenes. As a result, they cannot adapt to continual learning in dynamic robotic settings. To address these challenges, we propose TACO (TemporAl Consensus Optimization), a replay-free framework for continual neural mapping. We reformulate mapping as a temporal consensus optimization problem, where we treat past model snapshots as temporal neighbors. Intuitively, our approach resembles a model consulting its own past knowledge. We update the current map by enforcing weighted consensus with historical representations. Our method allows reliable past geometry to constrain optimization while permitting unreliable or outdated regions to be revised in response to new observations. TACO achieves a balance between memory efficiency and adaptability without storing or replaying previous data. Through extensive simulated and real-world experiments, we show that TACO robustly adapts to scene changes, and consistently outperforms other continual learning baselines.