Task Planning for Object Rearrangement in Multi-room Environments

TL;DR

This paper introduces a hierarchical task planner that combines language models, reinforcement learning, and novel algorithms to efficiently plan object rearrangement in multi-room environments, reducing steps and travel.

Contribution

It presents a new hierarchical planning approach with techniques for unseen object discovery, collision handling, and scalable state representation, along with a benchmark dataset for evaluation.

Findings

The proposed method outperforms existing approaches in multi-room rearrangement tasks.

It effectively reduces the number of steps and agent travel distance.

The new benchmark dataset MoPOR enables comprehensive evaluation.

Abstract

Object rearrangement in a multi-room setup should produce a reasonable plan that reduces the agent's overall travel and the number of steps. Recent state-of-the-art methods fail to produce such plans because they rely on explicit exploration for discovering unseen objects due to partial observability and a heuristic planner to sequence the actions for rearrangement. This paper proposes a novel hierarchical task planner to efficiently plan a sequence of actions to discover unseen objects and rearrange misplaced objects within an untidy house to achieve a desired tidy state. The proposed method introduces several novel techniques, including (i) a method for discovering unseen objects using commonsense knowledge from large language models, (ii) a collision resolution and buffer prediction method based on Cross-Entropy Method to handle blocked goal and swap cases, (iii) a directed spatial graph-based state space for scalability, and (iv) deep reinforcement learning (RL) for producing an efficient planner. The planner interleaves the discovery of unseen objects and rearrangement to minimize the number of steps taken and overall traversal of the agent. The paper also presents new metrics and a benchmark dataset called MoPOR to evaluate the effectiveness of the rearrangement planning in a multi-room setting. The experimental results demonstrate that the proposed method effectively addresses the multi-room rearrangement problem.