Self-Improving Semantic Perception for Indoor Localisation

TL;DR

This paper introduces a robotic system capable of online self-improvement in semantic perception and localization during deployment through continual learning and self-supervision, adapting to different environments without external data.

Contribution

It presents a novel framework for continuous, online semantic model updating on robots, enhancing localization accuracy across diverse environments without external supervision.

Findings

Semantic perception improved by 60% in segmentation accuracy.

Localization accuracy increased by 10%.

Memory replay reduces catastrophic forgetting.

Abstract

We propose a novel robotic system that can improve its perception during deployment. Contrary to the established approach of learning semantics from large datasets and deploying fixed models, we propose a framework in which semantic models are continuously updated on the robot to adapt to the deployment environments. By combining continual learning with self-supervision, our robotic system learns online during deployment without external supervision. We conduct real-world experiments with robots localising in 3D floorplans. Our experiments show how the robot's semantic perception improves during deployment and how this translates into improved localisation, even across drastically different environments. We further study the risk of catastrophic forgetting that such a continuous learning setting poses. We find memory replay an effective measure to reduce forgetting and show how the robotic system can improve even when switching between different environments. On average, our system improves by 60% in segmentation and 10% in localisation accuracy compared to deployment of a fixed model, and it maintains this improvement while adapting to further environments.