The Empirical Impact of Forgetting and Transfer in Continual Visual Odometry

TL;DR

This paper empirically examines how forgetting and transfer affect neural network performance in continual visual odometry tasks for embodied agents, highlighting challenges in balancing adaptation and memory retention.

Contribution

It provides the first empirical analysis of catastrophic forgetting and knowledge transfer in lifelong visual odometry within embodied robotics, revealing limitations of regularization and capacity strategies.

Findings

Transferability is high initially but decreases during specialization.

Regularization and increased capacity are ineffective in preventing forgetting.

Rehearsal offers mild benefits at high memory costs.

Abstract

As robotics continues to advance, the need for adaptive and continuously-learning embodied agents increases, particularly in the realm of assistance robotics. Quick adaptability and long-term information retention are essential to operate in dynamic environments typical of humans' everyday lives. A lifelong learning paradigm is thus required, but it is scarcely addressed by current robotics literature. This study empirically investigates the impact of catastrophic forgetting and the effectiveness of knowledge transfer in neural networks trained continuously in an embodied setting. We focus on the task of visual odometry, which holds primary importance for embodied agents in enabling their self-localization. We experiment on the simple continual scenario of discrete transitions between indoor locations, akin to a robot navigating different apartments. In this regime, we observe initial satisfactory performance with high transferability between environments, followed by a specialization phase where the model prioritizes current environment-specific knowledge at the expense of generalization. Conventional regularization strategies and increased model capacity prove ineffective in mitigating this phenomenon. Rehearsal is instead mildly beneficial but with the addition of a substantial memory cost. Incorporating action information, as commonly done in embodied settings, facilitates quicker convergence but exacerbates specialization, making the model overly reliant on its motion expectations and less adept at correctly interpreting visual cues. These findings emphasize the open challenges of balancing adaptation and memory retention in lifelong robotics and contribute valuable insights into the application of a lifelong paradigm on embodied agents.