Towards Lifelong Federated Learning in Autonomous Mobile Robots with Continuous Sim-to-Real Transfer

TL;DR

This paper explores federated learning for vision-based obstacle avoidance in mobile robots, demonstrating continuous model improvement through simulation and real-world data without compromising privacy.

Contribution

It introduces a federated learning approach for mobile robots that enables continuous, privacy-preserving model updates from both simulated and real environments.

Findings

Federated learning outperforms centralized training in accuracy.

Continuous learning from simulation and real-world data improves model performance.

Automatically labeled data from extended sensors enhances training efficiency.

Abstract

The role of deep learning (DL) in robotics has significantly deepened over the last decade. Intelligent robotic systems today are highly connected systems that rely on DL for a variety of perception, control, and other tasks. At the same time, autonomous robots are being increasingly deployed as part of fleets, with collaboration among robots becoming a more relevant factor. From the perspective of collaborative learning, federated learning (FL) enables continuous training of models in a distributed, privacy-preserving way. This paper focuses on vision-based obstacle avoidance for mobile robot navigation. On this basis, we explore the potential of FL for distributed systems of mobile robots enabling continuous learning via the engagement of robots in both simulated and real-world scenarios. We extend previous works by studying the performance of different image classifiers for FL, compared to centralized, cloud-based learning with a priori aggregated data. We also introduce an approach to continuous learning from mobile robots with extended sensor suites able to provide automatically labeled data while they are completing other tasks. We show that higher accuracies can be achieved by training the models in both simulation and reality, enabling continuous updates to deployed models.