Shape-Space Deformer: Unified Visuo-Tactile Representations for Robotic Manipulation of Deformable Objects

TL;DR

Shape-Space Deformer introduces a unified visuo-tactile representation that enhances robotic manipulation of deformable objects by improving generalization, robustness, and real-time reconstruction accuracy for unseen forces and objects.

Contribution

It proposes a novel Shape-Space Deformer model with template augmentation, enabling robust, fine-grained, and generalizable deformation modeling for robotic manipulation.

Findings

Outperforms existing methods in deformation reconstruction accuracy.

Demonstrates robustness to outliers and unseen forces.

Achieves real-time performance suitable for manipulation tasks.

Abstract

Accurate modelling of object deformations is crucial for a wide range of robotic manipulation tasks, where interacting with soft or deformable objects is essential. Current methods struggle to generalise to unseen forces or adapt to new objects, limiting their utility in real-world applications. We propose Shape-Space Deformer, a unified representation for encoding a diverse range of object deformations using template augmentation to achieve robust, fine-grained reconstructions that are resilient to outliers and unwanted artefacts. Our method improves generalization to unseen forces and can rapidly adapt to novel objects, significantly outperforming existing approaches. We perform extensive experiments to test a range of force generalisation settings and evaluate our method's ability to reconstruct unseen deformations, demonstrating significant improvements in reconstruction accuracy and robustness. Our approach is suitable for real-time performance, making it ready for downstream manipulation applications.