Disentangling perception and reasoning for improving data efficiency in learning cloth manipulation without demonstrations

TL;DR

This paper introduces a modular, efficient reinforcement learning approach for cloth manipulation that reduces model size and training time, enabling effective sim-to-real transfer and outperforming existing methods on benchmark tasks.

Contribution

It presents a novel, scalable RL framework that disentangles perception and reasoning, significantly improving data efficiency and transferability in cloth manipulation tasks.

Findings

Reduced model size and training time in simulation.

Successful transfer of policies to real-world cloth manipulation.

Outperformed baseline methods on SoftGym benchmark.

Abstract

Cloth manipulation is a ubiquitous task in everyday life, but it remains an open challenge for robotics. The difficulties in developing cloth manipulation policies are attributed to the high-dimensional state space, complex dynamics, and high propensity to self-occlusion exhibited by fabrics. As analytical methods have not been able to provide robust and general manipulation policies, reinforcement learning (RL) is considered a promising approach to these problems. However, to address the large state space and complex dynamics, data-based methods usually rely on large models and long training times. The resulting computational cost significantly hampers the development and adoption of these methods. Additionally, due to the challenge of robust state estimation, garment manipulation policies often adopt an end-to-end learning approach with workspace images as input. While this approach enables a conceptually straightforward sim-to-real transfer via real-world fine-tuning, it also incurs a significant computational cost by training agents on a highly lossy representation of the environment state. This paper questions this common design choice by exploring an efficient and modular approach to RL for cloth manipulation. We show that, through careful design choices, model size and training time can be significantly reduced when learning in simulation. Furthermore, we demonstrate how the resulting simulation-trained model can be transferred to the real world. We evaluate our approach on the SoftGym benchmark and achieve significant performance improvements over available baselines on our task, while using a substantially smaller model.