PerAct2: Benchmarking and Learning for Robotic Bimanual Manipulation Tasks

TL;DR

This paper introduces a comprehensive benchmark and a novel learning method for bimanual robotic manipulation, enabling systematic evaluation and learning of complex tasks involving two arms.

Contribution

It extends RLBench with a new bimanual benchmark and proposes PerAct2, a language-conditioned behavioral cloning approach for learning bimanual manipulation tasks.

Findings

Extended RLBench with 13 new bimanual tasks and 23 variations.

PerAct2 effectively learns complex bimanual tasks from language instructions.

The proposed architecture integrates language understanding with action prediction.

Abstract

Bimanual manipulation is challenging due to precise spatial and temporal coordination required between two arms. While there exist several real-world bimanual systems, there is a lack of simulated benchmarks with a large task diversity for systematically studying bimanual capabilities across a wide range of tabletop tasks. This paper addresses the gap by extending RLBench to bimanual manipulation. We open-source our code and benchmark comprising 13 new tasks with 23 unique task variations, each requiring a high degree of coordination and adaptability. To kickstart the benchmark, we extended several state-of-the art methods to bimanual manipulation and also present a language-conditioned behavioral cloning agent -- PerAct2, which enables the learning and execution of bimanual 6-DoF manipulation tasks. Our novel network architecture efficiently integrates language processing with action prediction, allowing robots to understand and perform complex bimanual tasks in response to user-specified goals. Project website with code is available at: http://bimanual.github.io