Intrinsically Motivated Multimodal Structure Learning

TL;DR

This paper introduces a long-term, intrinsically motivated method for learning multimodal structure and transition models of semi-permanent objects in the environment, enabling better planning and interaction in robotics.

Contribution

It presents a novel structure learning approach that models transition dynamics using distributions over partially observable states, specifically designed for multimodal affordance representation in robotics.

Findings

Model acquisition improves with more transition actions.

The approach effectively predicts state changes in response to interactions.

Experiments demonstrate the method's applicability on a mobile manipulator.

Abstract

We present a long-term intrinsically motivated structure learning method for modeling transition dynamics during controlled interactions between a robot and semi-permanent structures in the world. In particular, we discuss how partially-observable state is represented using distributions over a Markovian state and build models of objects that predict how state distributions change in response to interactions with such objects. These structures serve as the basis for a number of possible future tasks defined as Markov Decision Processes (MDPs). The approach is an example of a structure learning technique applied to a multimodal affordance representation that yields a population of forward models for use in planning. We evaluate the approach using experiments on a bimanual mobile manipulator (uBot-6) that show the performance of model acquisition as the number of transition actions increases.