Autonomous Open-Ended Learning of Tasks with Non-Stationary Interdependencies

TL;DR

This paper presents H-GRAIL, a hierarchical system enabling autonomous learning of interdependent, non-stationary tasks in robots by integrating goal relationships and adaptive strategies for task selection.

Contribution

It introduces a hierarchical architecture that models goal interdependencies as a Markov Decision Process and extends it with a learning layer to adapt to non-stationary task relationships.

Findings

H-GRAIL effectively learns interdependent skills in robotic tasks.

Incorporating task relationships improves goal selection efficiency.

The system adapts to changing task interdependencies in real-time.

Abstract

Autonomous open-ended learning is a relevant approach in machine learning and robotics, allowing the design of artificial agents able to acquire goals and motor skills without the necessity of user assigned tasks. A crucial issue for this approach is to develop strategies to ensure that agents can maximise their competence on as many tasks as possible in the shortest possible time. Intrinsic motivations have proven to generate a task-agnostic signal to properly allocate the training time amongst goals. While the majority of works in the field of intrinsically motivated open-ended learning focus on scenarios where goals are independent from each other, only few of them studied the autonomous acquisition of interdependent tasks, and even fewer tackled scenarios where goals involve non-stationary interdependencies. Building on previous works, we tackle these crucial issues at the level of decision making (i.e., building strategies to properly select between goals), and we propose a hierarchical architecture that treating sub-tasks selection as a Markov Decision Process is able to properly learn interdependent skills on the basis of intrinsically generated motivations. In particular, we first deepen the analysis of a previous system, showing the importance of incorporating information about the relationships between tasks at a higher level of the architecture (that of goal selection). Then we introduce H-GRAIL, a new system that extends the previous one by adding a new learning layer to store the autonomously acquired sequences of tasks to be able to modify them in case the interdependencies are non-stationary. All systems are tested in a real robotic scenario, with a Baxter robot performing multiple interdependent reaching tasks.