Omnipush: accurate, diverse, real-world dataset of pushing dynamics with RGB-D video

TL;DR

Omnipush is a comprehensive RGB-D dataset of 250 objects with 250 pushes each, designed to facilitate learning and benchmarking of diverse and accurate pushing dynamics models in robotics.

Contribution

The paper introduces Omnipush, a novel dataset capturing diverse pushing behaviors with systematic variation in object shape and mass distribution, enabling better generalization and model evaluation.

Findings

Provides high-quality RGB-D push data for 250 objects

Includes benchmarks for meta-learning dynamic models

Facilitates research on generalization in pushing dynamics

Abstract

Pushing is a fundamental robotic skill. Existing work has shown how to exploit models of pushing to achieve a variety of tasks, including grasping under uncertainty, in-hand manipulation and clearing clutter. Such models, however, are approximate, which limits their applicability. Learning-based methods can reason directly from raw sensory data with accuracy, and have the potential to generalize to a wider diversity of scenarios. However, developing and testing such methods requires rich-enough datasets. In this paper we introduce Omnipush, a dataset with high variety of planar pushing behavior. In particular, we provide 250 pushes for each of 250 objects, all recorded with RGB-D and a high precision tracking system. The objects are constructed so as to systematically explore key factors that affect pushing -- the shape of the object and its mass distribution -- which have not been broadly explored in previous datasets, and allow to study generalization in model learning. Omnipush includes a benchmark for meta-learning dynamic models, which requires algorithms that make good predictions and estimate their own uncertainty. We also provide an RGB video prediction benchmark and propose other relevant tasks that can be suited with this dataset. Data and code are available at \url{https://web.mit.edu/mcube/omnipush-dataset/}.