# Monte-Carlo Tree Search for Efficient Visually Guided Rearrangement Planning

## TL;DR

This paper presents a scalable Monte-Carlo Tree Search-based method for visually guided rearrangement planning with many objects, combined with a neural network for robust object localization from RGB images, enabling efficient real-time robotic manipulation.

## Contribution

It introduces a novel, scalable rearrangement planning approach using Monte-Carlo Tree Search and a deep learning-based multi-object localization from RGB images, without requiring buffer space.

## Key findings

- Scales well with up to 25 objects in real-time.
- Finds solutions with fewer moves than state-of-the-art methods.
- Robust to camera movements and external perturbations.

## Abstract

We address the problem of visually guided rearrangement planning with many movable objects, i.e., finding a sequence of actions to move a set of objects from an initial arrangement to a desired one, while relying on visual inputs coming from an RGB camera. To do so, we introduce a complete pipeline relying on two key contributions. First, we introduce an efficient and scalable rearrangement planning method, based on a Monte-Carlo Tree Search exploration strategy. We demonstrate that because of its good trade-off between exploration and exploitation our method (i) scales well with the number of objects while (ii) finding solutions which require a smaller number of moves compared to the other state-of-the-art approaches. Note that on the contrary to many approaches, we do not require any buffer space to be available. Second, to precisely localize movable objects in the scene, we develop an integrated approach for robust multi-object workspace state estimation from a single uncalibrated RGB camera using a deep neural network trained only with synthetic data. We validate our multi-object visually guided manipulation pipeline with several experiments on a real UR-5 robotic arm by solving various rearrangement planning instances, requiring only 60 ms to compute the plan to rearrange 25 objects. In addition, we show that our system is insensitive to camera movements and can successfully recover from external perturbations. Supplementary video, source code and pre-trained models are available at https://ylabbe.github.io/rearrangement-planning.