GarNet: A Continuous Robot Vision Approach for Predicting Shapes and Visually Perceived Weights of Garments

TL;DR

GarNet is a novel robot vision system that learns to predict garment shapes and perceived weights by observing garments during pickup, enabling accurate classification without full observation of the garment's state.

Contribution

Introduces GarNet, a continuous observation approach that predicts garment shape and weight efficiently, outperforming existing methods by 21% in shape classification accuracy.

Findings

Achieves 92% accuracy in shape classification.

Predicts perceived weight with 95.5% accuracy.

Outperforms state-of-the-art by 21% in shape classification.

Abstract

We present a Garment Similarity Network (GarNet) that learns geometric and physical similarities between known garments by continuously observing a garment while a robot picks it up from a table. The aim is to capture and encode geometric and physical characteristics of a garment into a manifold where a decision can be carried out, such as predicting the garment's shape class and its visually perceived weight. Our approach features an early stop strategy, which means that GarNet does not need to observe a garment being picked up from a crumpled to a hanging state to make a prediction. In our experiments, we find that GarNet achieves prediction accuracies of 92% for shape classification and 95.5% for predicting weights and advances state-of-art approaches by 21% for shape classification.