DiffClone: Enhanced Behaviour Cloning in Robotics with Diffusion-Driven Policy Learning

TL;DR

DiffClone introduces a diffusion-based behaviour cloning algorithm for robot manipulation, demonstrating improved success rates and rewards on real robots within the TOTO benchmark, leveraging pre-trained visual features.

Contribution

We propose DiffClone, a novel offline behaviour cloning method utilizing diffusion models for policy learning, and evaluate its effectiveness on real robots in the TOTO benchmark.

Findings

MOCO finetuned ResNet50 outperforms other visual representations.

Goal conditioning and transition mapping slightly improve success and reward.

DiffClone with diffusion-based policy learning enhances robot manipulation performance.

Abstract

Robot learning tasks are extremely compute-intensive and hardware-specific. Thus the avenues of tackling these challenges, using a diverse dataset of offline demonstrations that can be used to train robot manipulation agents, is very appealing. The Train-Offline-Test-Online (TOTO) Benchmark provides a well-curated open-source dataset for offline training comprised mostly of expert data and also benchmark scores of the common offline-RL and behaviour cloning agents. In this paper, we introduce DiffClone, an offline algorithm of enhanced behaviour cloning agent with diffusion-based policy learning, and measured the efficacy of our method on real online physical robots at test time. This is also our official submission to the Train-Offline-Test-Online (TOTO) Benchmark Challenge organized at NeurIPS 2023. We experimented with both pre-trained visual representation and agent policies. In our experiments, we find that MOCO finetuned ResNet50 performs the best in comparison to other finetuned representations. Goal state conditioning and mapping to transitions resulted in a minute increase in the success rate and mean-reward. As for the agent policy, we developed DiffClone, a behaviour cloning agent improved using conditional diffusion.