Hierarchical Policy for Non-prehensile Multi-object Rearrangement with Deep Reinforcement Learning and Monte Carlo Tree Search

TL;DR

This paper introduces a hierarchical reinforcement learning and Monte Carlo Tree Search approach for complex multi-object rearrangement tasks in robotics, improving success rates and efficiency over existing methods.

Contribution

It presents a novel hierarchical policy combining MCTS and deep learning for non-prehensile multi-object rearrangement, addressing planning complexity.

Findings

Higher success rate compared to state-of-the-art methods

Fewer steps and shorter paths in rearrangement tasks

Effective integration of imitation and reinforcement learning

Abstract

Non-prehensile multi-object rearrangement is a robotic task of planning feasible paths and transferring multiple objects to their predefined target poses without grasping. It needs to consider how each object reaches the target and the order of object movement, which significantly deepens the complexity of the problem. To address these challenges, we propose a hierarchical policy to divide and conquer for non-prehensile multi-object rearrangement. In the high-level policy, guided by a designed policy network, the Monte Carlo Tree Search efficiently searches for the optimal rearrangement sequence among multiple objects, which benefits from imitation and reinforcement. In the low-level policy, the robot plans the paths according to the order of path primitives and manipulates the objects to approach the goal poses one by one. We verify through experiments that the proposed method can achieve a higher success rate, fewer steps, and shorter path length compared with the state-of-the-art.