Exploration with Intrinsic Motivation using Object-Action-Outcome Latent Space

TL;DR

This paper introduces an intrinsic motivation-based exploration method that organizes robot learning in a latent space combining actions, objects, and outcomes, leading to faster learning and developmental stages similar to infants.

Contribution

It proposes a novel exploration mechanism using a combined latent space and intrinsic motivation to improve robot learning efficiency and developmental progression.

Findings

Outperforms existing intrinsic motivation approaches in learning speed

Organizes learning curriculum similar to infant development stages

Learns to predict outcomes of skills in a staged manner

Abstract

One effective approach for equipping artificial agents with sensorimotor skills is to use self-exploration. To do this efficiently is critical, as time and data collection are costly. In this study, we propose an exploration mechanism that blends action, object, and action outcome representations into a latent space, where local regions are formed to host forward model learning. The agent uses intrinsic motivation to select the forward model with the highest learning progress to adopt at a given exploration step. This parallels how infants learn, as high learning progress indicates that the learning problem is neither too easy nor too difficult in the selected region. The proposed approach is validated with a simulated robot in a table-top environment. The simulation scene comprises a robot and various objects, where the robot interacts with one of them each time using a set of parameterized actions and learns the outcomes of these interactions. With the proposed approach, the robot organizes its curriculum of learning as in existing intrinsic motivation approaches and outperforms them in learning speed. Moreover, the learning regime demonstrates features that partially match infant development; in particular, the proposed system learns to predict the outcomes of different skills in a staged manner.