Benchmarking Robot Manipulation with the Rubik's Cube

TL;DR

This paper introduces a standardized Rubik's cube manipulation benchmark for robots, enabling measurement of manipulation accuracy and speed across different platforms and approaches, demonstrated on PR2 and HERB robots.

Contribution

It proposes a new, accessible benchmark for robot manipulation skills using Rubik's cube, with protocols and baseline implementations for various robots and algorithms.

Findings

Baseline approaches demonstrate the protocol's effectiveness.

Integration of pre-touch sensing improves performance.

The benchmark is adaptable to different robot platforms.

Abstract

Benchmarks for robot manipulation are crucial to measuring progress in the field, yet there are few benchmarks that demonstrate critical manipulation skills, possess standardized metrics, and can be attempted by a wide array of robot platforms. To address a lack of such benchmarks, we propose Rubik's cube manipulation as a benchmark to measure simultaneous performance of precise manipulation and sequential manipulation. The sub-structure of the Rubik's cube demands precise positioning of the robot's end effectors, while its highly reconfigurable nature enables tasks that require the robot to manage pose uncertainty throughout long sequences of actions. We present a protocol for quantitatively measuring both the accuracy and speed of Rubik's cube manipulation. This protocol can be attempted by any general-purpose manipulator, and only requires a standard 3x3 Rubik's cube and a flat surface upon which the Rubik's cube initially rests (e.g. a table). We demonstrate this protocol for two distinct baseline approaches on a PR2 robot. The first baseline provides a fundamental approach for pose-based Rubik's cube manipulation. The second baseline demonstrates the benchmark's ability to quantify improved performance by the system, particularly that resulting from the integration of pre-touch sensing. To demonstrate the benchmark's applicability to other robot platforms and algorithmic approaches, we present the functional blocks required to enable the HERB robot to manipulate the Rubik's cube via push-grasping.