Decomposed Object Manipulation via Dual-Actor Policy

TL;DR

This paper introduces a Dual-Actor Policy (DAP) for object manipulation that explicitly models different stages of the task, leveraging visual priors and a decision mechanism to improve performance across simulated and real-world scenarios.

Contribution

The paper proposes a novel Dual-Actor Policy that explicitly separates approaching and manipulation stages, utilizing heterogeneous visual priors and a decision maker for enhanced object manipulation.

Findings

Outperforms state-of-the-art by 5.55%, 14.7%, and 10.4% on various benchmarks.

Constructs a new dataset with visual priors and multi-stage tasks.

Demonstrates effectiveness in both simulation and real-world environments.

Abstract

Object manipulation, which focuses on learning to perform tasks on similar parts across different types of objects, can be divided into an approaching stage and a manipulation stage. However, previous works often ignore this characteristic of the task and rely on a single policy to directly learn the whole process of object manipulation. To address this problem, we propose a novel Dual-Actor Policy, termed DAP, which explicitly considers different stages and leverages heterogeneous visual priors to enhance each stage. Specifically, we introduce an affordance-based actor to locate the functional part in the manipulation task, thereby improving the approaching process. Following this, we propose a motion flow-based actor to capture the movement of the component, facilitating the manipulation process. Finally, we introduce a decision maker to determine the current stage of DAP and select the corresponding actor. Moreover, existing object manipulation datasets contain few objects and lack the visual priors needed to support training. To address this, we construct a simulated dataset, the Dual-Prior Object Manipulation Dataset, which combines the two visual priors and includes seven tasks, including two challenging long-term, multi-stage tasks. Experimental results on our dataset, the RoboTwin benchmark and real-world scenarios illustrate that our method consistently outperforms the SOTA method by 5.55%, 14.7% and 10.4% on average respectively.