Function based sim-to-real learning for shape control of deformable free-form surfaces

TL;DR

This paper introduces a novel deformation function based sim-to-real learning approach that effectively maps simulated shapes to real-world deformable surfaces, accommodating sparse data and marker loss, improving shape control accuracy.

Contribution

It presents a new sim-to-real learning method that handles sparse markers and missing data, enhancing deformable surface shape control in practical applications.

Findings

Effective mapping from simulation to real shapes demonstrated

Resilient to missing markers and sparse data

Integrated into a neural network pipeline for inverse kinematics

Abstract

For the shape control of deformable free-form surfaces, simulation plays a crucial role in establishing the mapping between the actuation parameters and the deformed shapes. The differentiation of this forward kinematic mapping is usually employed to solve the inverse kinematic problem for determining the actuation parameters that can realize a target shape. However, the free-form surfaces obtained from simulators are always different from the physically deformed shapes due to the errors introduced by hardware and the simplification adopted in physical simulation. To fill the gap, we propose a novel deformation function based sim-to-real learning method that can map the geometric shape of a simulated model into its corresponding shape of the physical model. Unlike the existing sim-to-real learning methods that rely on completely acquired dense markers, our method accommodates sparsely distributed markers and can resiliently use all captured frames -- even for those in the presence of missing markers. To demonstrate its effectiveness, our sim-to-real method has been integrated into a neural network-based computational pipeline designed to tackle the inverse kinematic problem on a pneumatically actuated deformable mannequin.