Subspace-wise Hybrid RL for Articulated Object Manipulation

TL;DR

This paper introduces a Subspace-wise hybrid RL framework that divides the task space into subspaces to improve learning efficiency and dexterity in articulated object manipulation, validated through simulations and real-world tests.

Contribution

The novel subspace-wise hybrid RL approach enables adaptive control and better utilization of redundant subspaces for articulated object manipulation.

Findings

Enhanced learning efficiency in simulated and real-world tasks

Improved dexterity through subspace exploitation

Validated effectiveness via experiments

Abstract

Articulated object manipulation is a challenging task, requiring constrained motion and adaptive control to handle the unknown dynamics of the manipulated objects. While reinforcement learning (RL) has been widely employed to tackle various scenarios and types of articulated objects, the complexity of these tasks, stemming from multiple intertwined objectives makes learning a control policy in the full task space highly difficult. To address this issue, we propose a Subspace-wise hybrid RL (SwRL) framework that learns policies for each divided task space, or subspace, based on independent objectives. This approach enables adaptive force modulation to accommodate the unknown dynamics of objects. Additionally, it effectively leverages the previously underlooked redundant subspace, thereby maximizing the robot's dexterity. Our method enhances both learning efficiency and task execution performance, as validated through simulations and real-world experiments. Supplementary video is available at https://youtu.be/PkNxv0P8Atk