Learning Cloth Folding Tasks with Refined Flow Based Spatio-Temporal Graphs

TL;DR

This paper introduces a novel learning approach for robotic cloth folding using refined flow-based spatiotemporal graphs, enabling better generalization and real-time handling of deformable textiles through demonstration-based learning.

Contribution

It presents a new high-level encoding method for cloth folding demonstrations using optical flow-based graphs, improving generalization and real-time performance in robotic manipulation.

Findings

Effective cloth folding demonstrated on real robot

Enhanced generalization across different environments

Real-time handling of system dynamics achieved

Abstract

Cloth folding is a widespread domestic task that is seemingly performed by humans but which is highly challenging for autonomous robots to execute due to the highly deformable nature of textiles; It is hard to engineer and learn manipulation pipelines to efficiently execute it. In this paper, we propose a new solution for robotic cloth folding (using a standard folding board) via learning from demonstrations. Our demonstration video encoding is based on a high-level abstraction, namely, a refined optical flow-based spatiotemporal graph, as opposed to a low-level encoding such as image pixels. By constructing a new spatiotemporal graph with an advanced visual corresponding descriptor, the policy learning can focus on key points and relations with a 3D spatial configuration, which allows to quickly generalize across different environments. To further boost the policy searching, we combine optical flow and static motion saliency maps to discriminate the dominant motions for better handling the system dynamics in real-time, which aligns with the attentional motion mechanism that dominates the human imitation process. To validate the proposed approach, we analyze the manual folding procedure and developed a custom-made end-effector to efficiently interact with the folding board. Multiple experiments on a real robotic platform were conducted to validate the effectiveness and robustness of the proposed method.