# Deep learning-based robotic cloth manipulation applications: systematic review, challenges and opportunities for physical AI

## TL;DR

This paper reviews recent deep learning-based robotic cloth manipulation methods, highlighting progress, challenges, and future directions in physical AI.

## Contribution

A systematic review of 41 papers on deep learning for cloth manipulation, categorizing methods into six learning paradigms and identifying key challenges.

## Key findings

- Current methods struggle with irregular cloth sizes and diverse initial garment states.
- Improved real-world data collection and realistic cloth simulators are needed to bridge the Sim2Real gap.
- Six learning and control paradigms are identified, each with strengths and limitations for cloth manipulation tasks.

## Abstract

Cloth unfolding and folding are fundamental tasks in autonomous robotic cloth manipulation as Physical AI. Driven by recent advances in deep learning, this area has developed rapidly in recent years. This review aims to systematically identify and summarize current progress in deep learning-based cloth unfolding and folding. Following the Systematic Reviews and Meta-Analyses (PRISMA) guidelines, 41 relevant papers from 2019 to 2024 were selected for analysis. We examines various factors influencing cloth manipulation and find that, while current methods show impressive performance, several challenges remain unaddressed. These challenges include irregular cloth sizes and diverse initial garment states. Concerning datasets, there is a need for improved real-world data collection systems and more realistic cloth simulators, and the Sim2Real gap must be carefully considered. Additionally, the review highlights the importance of incorporating multi-modal sensors into current platforms and the emergence of novel primitive actions that enhance performance. The need for more consistent comparison metrics is emphasized, and strategies for addressing failure modes are discussed to further advance the field. From an algorithmic perspective, we reorganize existing learning methods into six learning and control paradigms: perception-guided heuristics, goal-conditioned manipulation policies, predictive and model-based state representation methods, reward-driven reinforcement learning over primitive actions, demonstration-driven skill transfer methods, and emerging large language model-based planning methods. We discuss how each paradigm contributes to unfolding and folding, their respective strengths and limitations, and the open problems that arise. Finally, we summarize the remaining challenges and provide future perspectives for physical AI.