Autotelic Reinforcement Learning: Exploring Intrinsic Motivations for Skill Acquisition in Open-Ended Environments

TL;DR

This paper reviews autotelic reinforcement learning, highlighting intrinsic motivations for autonomous skill development and goal exploration in open-ended environments, and discusses evaluation metrics for such agents.

Contribution

It provides a comprehensive overview of autotelic RL, clarifies motivation types, and introduces a framework for goal exploration and evaluation in reward-free settings.

Findings

Distinction between knowledge-based and competence-based motivations

Typology of Intrinsically Motivated Goal Exploration Processes (IMGEPs)

Proposed metrics for evaluating exploration and generalization

Abstract

This paper presents a comprehensive overview of autotelic Reinforcement Learning (RL), emphasizing the role of intrinsic motivations in the open-ended formation of skill repertoires. We delineate the distinctions between knowledge-based and competence-based intrinsic motivations, illustrating how these concepts inform the development of autonomous agents capable of generating and pursuing self-defined goals. The typology of Intrinsically Motivated Goal Exploration Processes (IMGEPs) is explored, with a focus on the implications for multi-goal RL and developmental robotics. The autotelic learning problem is framed within a reward-free Markov Decision Process (MDP), WHERE agents must autonomously represent, generate, and master their own goals. We address the unique challenges in evaluating such agents, proposing various metrics for measuring exploration, generalization, and robustness in complex environments. This work aims to advance the understanding of autotelic RL agents and their potential for enhancing skill acquisition in a diverse and dynamic setting.