CoVIO: Online Continual Learning for Visual-Inertial Odometry

TL;DR

CoVIO introduces an online continual learning approach for visual-inertial odometry that adapts to new environments in real-time, mitigates forgetting, and is suitable for embedded devices, demonstrated through extensive real-world evaluations.

Contribution

We propose CoVIO, a novel online continual learning framework for visual-inertial odometry that uses a new sampling strategy and asynchronous updates for real-time domain adaptation.

Findings

Successfully adapts to new domains in real-world datasets

Outperforms previous methods in continual learning scenarios

Effective in resource-constrained embedded environments

Abstract

Visual odometry is a fundamental task for many applications on mobile devices and robotic platforms. Since such applications are oftentimes not limited to predefined target domains and learning-based vision systems are known to generalize poorly to unseen environments, methods for continual adaptation during inference time are of significant interest. In this work, we introduce CoVIO for online continual learning of visual-inertial odometry. CoVIO effectively adapts to new domains while mitigating catastrophic forgetting by exploiting experience replay. In particular, we propose a novel sampling strategy to maximize image diversity in a fixed-size replay buffer that targets the limited storage capacity of embedded devices. We further provide an asynchronous version that decouples the odometry estimation from the network weight update step enabling continuous inference in real time. We extensively evaluate CoVIO on various real-world datasets demonstrating that it successfully adapts to new domains while outperforming previous methods. The code of our work is publicly available at http://continual-slam.cs.uni-freiburg.de.