Improving Behavioural Cloning with Positive Unlabeled Learning

TL;DR

This paper introduces a new iterative algorithm that identifies high-quality expert trajectories from mixed datasets with minimal positive examples, improving offline policy learning in robotics.

Contribution

A novel iterative learning method for extracting expert trajectories from unlabeled mixed-quality data, enhancing behavioral cloning performance.

Findings

Outperforms existing algorithms in identifying expert trajectories.

Achieves state-of-the-art results on simulated and real robotic tasks.

Surpasses several offline reinforcement and imitation learning baselines.

Abstract

Learning control policies offline from pre-recorded datasets is a promising avenue for solving challenging real-world problems. However, available datasets are typically of mixed quality, with a limited number of the trajectories that we would consider as positive examples; i.e., high-quality demonstrations. Therefore, we propose a novel iterative learning algorithm for identifying expert trajectories in unlabeled mixed-quality robotics datasets given a minimal set of positive examples, surpassing existing algorithms in terms of accuracy. We show that applying behavioral cloning to the resulting filtered dataset outperforms several competitive offline reinforcement learning and imitation learning baselines. We perform experiments on a range of simulated locomotion tasks and on two challenging manipulation tasks on a real robotic system; in these experiments, our method showcases state-of-the-art performance. Our website: \url{https://sites.google.com/view/offline-policy-learning-pubc}.