Rearrangement: A Challenge for Embodied AI

TL;DR

This paper introduces a standardized framework and benchmark for the rearrangement task in Embodied AI, aiming to advance research by providing testbeds, metrics, and transferability of trained models across environments.

Contribution

It proposes a canonical rearrangement task, defines evaluation metrics, and provides simulation testbeds to facilitate research and development in Embodied AI.

Findings

Benchmark scenarios in four simulation environments

Metrics for evaluating rearrangement performance

Potential for transferring trained models to physical systems

Abstract

We describe a framework for research and evaluation in Embodied AI. Our proposal is based on a canonical task: Rearrangement. A standard task can focus the development of new techniques and serve as a source of trained models that can be transferred to other settings. In the rearrangement task, the goal is to bring a given physical environment into a specified state. The goal state can be specified by object poses, by images, by a description in language, or by letting the agent experience the environment in the goal state. We characterize rearrangement scenarios along different axes and describe metrics for benchmarking rearrangement performance. To facilitate research and exploration, we present experimental testbeds of rearrangement scenarios in four different simulation environments. We anticipate that other datasets will be released and new simulation platforms will be built to support training of rearrangement agents and their deployment on physical systems.