Keyframe Demonstration Seeded and Bayesian Optimized Policy Search

TL;DR

This paper presents a new Learning from Demonstration framework that combines keyframe demonstrations, Dynamic Bayesian Networks, and Bayesian Optimized Policy Search to enhance robotic skill learning and exploration efficiency.

Contribution

It introduces BO-PI2, a Bayesian optimized policy search method that leverages perceptual relations and reward prediction to improve learning from demonstrations.

Findings

BO-PI2 outperforms state-of-the-art methods in real robot tasks.

The approach effectively focuses exploration on failed sub-goals.

Increased reward and success rates demonstrate improved learning efficiency.

Abstract

This paper introduces a novel Learning from Demonstration framework to learn robotic skills with keyframe demonstrations using a Dynamic Bayesian Network (DBN) and a Bayesian Optimized Policy Search approach to improve the learned skills. DBN learns the robot motion, perceptual change in the object of interest (aka skill sub-goals) and the relation between them. The rewards are also learned from the perceptual part of the DBN. The policy search part is a semiblack box algorithm, which we call BO-PI2 . It utilizes the action-perception relation to focus the high-level exploration, uses Gaussian Processes to model the expected-return and performs Upper Confidence Bound type low-level exploration for sampling the rollouts. BO-PI2 is compared against a stateof-the-art method on three different skills in a real robot setting with expert and naive user demonstrations. The results show that our approach successfully focuses the exploration on the failed sub-goals and the addition of reward-predictive exploration outperforms the state-of-the-art approach on cumulative reward, skill success, and termination time metrics.