Learning from demonstration using products of experts: applications to manipulation and task prioritization

TL;DR

This paper introduces a product of experts framework for learning from demonstration that effectively combines multiple task space models, improving manipulation and task prioritization, especially for hierarchical objectives.

Contribution

It proposes a novel PoE-based method for joint learning of task models in LfD, with a variational inference approach, extending to nullspace structures for hierarchical tasks.

Findings

Joint learning in PoE improves model quality.

The approach handles hierarchical and competitive tasks effectively.

Variational inference offers a practical alternative to contrastive divergence.

Abstract

Probability distributions are key components of many learning from demonstration (LfD) approaches. While the configuration of a manipulator is defined by its joint angles, poses are often best explained within several task spaces. In many approaches, distributions within relevant task spaces are learned independently and only combined at the control level. This simplification implies several problems that are addressed in this work. We show that the fusion of models in different task spaces can be expressed as a product of experts (PoE), where the probabilities of the models are multiplied and renormalized so that it becomes a proper distribution of joint angles. Multiple experiments are presented to show that learning the different models jointly in the PoE framework significantly improves the quality of the model. The proposed approach particularly stands out when the robot has to learn competitive or hierarchical objectives. Training the model jointly usually relies on contrastive divergence, which requires costly approximations that can affect performance. We propose an alternative strategy using variational inference and mixture model approximations. In particular, we show that the proposed approach can be extended to PoE with a nullspace structure (PoENS), where the model is able to recover tasks that are masked by the resolution of higher-level objectives.