BulletArm: An Open-Source Robotic Manipulation Benchmark and Learning Framework

TL;DR

BulletArm is an open-source, extensible benchmark and learning framework for robotic manipulation, featuring 31 tasks and baseline algorithms to promote standardized evaluation and comparison of robotic learning methods.

Contribution

It introduces BulletArm, a reproducible and extensible simulation environment with diverse manipulation tasks and integrated baseline algorithms for benchmarking robotic learning.

Findings

Evaluation of five benchmarks with state-of-the-art algorithms.

Demonstrated the framework's extensibility and ease of adding new tasks.

Provided standardized tools for comparison of robotic manipulation methods.

Abstract

We present BulletArm, a novel benchmark and learning-environment for robotic manipulation. BulletArm is designed around two key principles: reproducibility and extensibility. We aim to encourage more direct comparisons between robotic learning methods by providing a set of standardized benchmark tasks in simulation alongside a collection of baseline algorithms. The framework consists of 31 different manipulation tasks of varying difficulty, ranging from simple reaching and picking tasks to more realistic tasks such as bin packing and pallet stacking. In addition to the provided tasks, BulletArm has been built to facilitate easy expansion and provides a suite of tools to assist users when adding new tasks to the framework. Moreover, we introduce a set of five benchmarks and evaluate them using a series of state-of-the-art baseline algorithms. By including these algorithms as part of our framework, we hope to encourage users to benchmark their work on any new tasks against these baselines.