Learning a Set of Interrelated Tasks by Using Sequences of Motor Policies for a Strategic Intrinsically Motivated Learner

TL;DR

This paper introduces an active learning framework for robots that learns complex, hierarchical tasks by autonomously discovering and combining motor policies through sequences called procedures, enhancing adaptability and skill composition.

Contribution

It presents a novel 'procedures' framework enabling robots to learn and combine simple skills into complex hierarchical tasks using active, goal-directed exploration strategies.

Findings

Successfully learned complex motor policies in simulation

Demonstrated effective hierarchical task learning with procedures

Adapted policy complexity to task difficulty

Abstract

We propose an active learning architecture for robots, capable of organizing its learning process to achieve a field of complex tasks by learning sequences of motor policies, called Intrinsically Motivated Procedure Babbling (IM-PB). The learner can generalize over its experience to continuously learn new tasks. It chooses actively what and how to learn based by empirical measures of its own progress. In this paper, we are considering the learning of a set of interrelated tasks outcomes hierarchically organized. We introduce a framework called 'procedures', which are sequences of policies defined by the combination of previously learned skills. Our algorithmic architecture uses the procedures to autonomously discover how to combine simple skills to achieve complex goals. It actively chooses between 2 strategies of goal-directed exploration : exploration of the policy space or the procedural space. We show on a simulated environment that our new architecture is capable of tackling the learning of complex motor policies, to adapt the complexity of its policies to the task at hand. We also show that our 'procedures' framework helps the learner to tackle difficult hierarchical tasks.