Behavior Self-Organization Supports Task Inference for Continual Robot Learning

TL;DR

This paper introduces a novel method for continual robot learning that uses unsupervised behavior embedding and self-organization to infer tasks without prior assumptions, improving generalization and learning speed.

Contribution

It presents a new approach combining behavior self-organization with reinforcement learning for task inference in continual robot learning, requiring no task exploration or distribution assumptions.

Findings

Outperforms existing multi-task learning methods in generalization.

Achieves faster convergence in continual learning scenarios.

Effectively infers tasks without prior task distribution knowledge.

Abstract

Recent advances in robot learning have enabled robots to become increasingly better at mastering a predefined set of tasks. On the other hand, as humans, we have the ability to learn a growing set of tasks over our lifetime. Continual robot learning is an emerging research direction with the goal of endowing robots with this ability. In order to learn new tasks over time, the robot first needs to infer the task at hand. Task inference, however, has received little attention in the multi-task learning literature. In this paper, we propose a novel approach to continual learning of robotic control tasks. Our approach performs unsupervised learning of behavior embeddings by incrementally self-organizing demonstrated behaviors. Task inference is made by finding the nearest behavior embedding to a demonstrated behavior, which is used together with the environment state as input to a multi-task policy trained with reinforcement learning to optimize performance over tasks. Unlike previous approaches, our approach makes no assumptions about task distribution and requires no task exploration to infer tasks. We evaluate our approach in experiments with concurrently and sequentially presented tasks and show that it outperforms other multi-task learning approaches in terms of generalization performance and convergence speed, particularly in the continual learning setting.