Identifying Expert Behavior in Offline Training Datasets Improves Behavioral Cloning of Robotic Manipulation Policies

TL;DR

This paper demonstrates that identifying expert data within mixed offline datasets and leveraging geometric symmetry can significantly improve behavioral cloning for robotic manipulation, surpassing advanced offline RL methods.

Contribution

The authors introduce a semi-supervised classifier to identify expert data in mixed datasets and utilize geometric symmetry for data augmentation, enhancing behavioral cloning performance.

Findings

Behavioral Cloning outperforms offline RL on expert datasets.

Identifying expert data improves policy learning from mixed datasets.

Data augmentation via symmetry boosts policy performance.

Abstract

This paper presents our solution for the Real Robot Challenge (RRC) III, a competition featured in the NeurIPS 2022 Competition Track, aimed at addressing dexterous robotic manipulation tasks through learning from pre-collected offline data. Participants were provided with two types of datasets for each task: expert and mixed datasets with varying skill levels. While the simplest offline policy learning algorithm, Behavioral Cloning (BC), performed remarkably well when trained on expert datasets, it outperformed even the most advanced offline reinforcement learning (RL) algorithms. However, BC's performance deteriorated when applied to mixed datasets, and the performance of offline RL algorithms was also unsatisfactory. Upon examining the mixed datasets, we observed that they contained a significant amount of expert data, although this data was unlabeled. To address this issue, we proposed a semi-supervised learning-based classifier to identify the underlying expert behavior within mixed datasets, effectively isolating the expert data. To further enhance BC's performance, we leveraged the geometric symmetry of the RRC arena to augment the training dataset through mathematical transformations. In the end, our submission surpassed that of all other participants, even those who employed complex offline RL algorithms and intricate data processing and feature engineering techniques.