Modal-graph 3D shape servoing of deformable objects with raw point clouds

TL;DR

This paper introduces a modal-graph framework for model-free 3D shape servoing of deformable objects using raw point clouds, effectively handling measurement noise and partial observability without complex preprocessing.

Contribution

The proposed method constructs a low-frequency deformation structure and a modal-graph representation for robust, model-free shape control directly from raw point clouds, avoiding registration and occlusion issues.

Findings

Effective shape servoing demonstrated on various object types.

Robustness to measurement noise and partial data shown in experiments.

Controller stability proven without offline learning.

Abstract

Deformable object manipulation (DOM) with point clouds has great potential as non-rigid 3D shapes can be measured without detecting and tracking image features. However, robotic shape control of deformable objects with point clouds is challenging due to: the unknown point-wise correspondences and the noisy partial observability of raw point clouds; the modeling difficulties of the relationship between point clouds and robot motions. To tackle these challenges, this paper introduces a novel modal-graph framework for the model-free shape servoing of deformable objects with raw point clouds. Unlike the existing works studying the object's geometry structure, our method builds a low-frequency deformation structure for the DOM system, which is robust to the measurement irregularities. The built modal representation and graph structure enable us to directly extract low-dimensional deformation features from raw point clouds. Such extraction requires no extra point processing of registrations, refinements, and occlusion removal. Moreover, to shape the object using the extracted features, we design an adaptive robust controller which is proved to be input-to-state stable (ISS) without offline learning or identifying both the physical and geometric object models. Extensive simulations and experiments are conducted to validate the effectiveness of our method for linear, planar, tubular, and solid objects under different settings.