Sensing and Reconstruction of 3D Deformation on Pneumatic Soft Robots

TL;DR

This paper presents a real-time method for sensing and reconstructing 3D deformation in pneumatic soft robots using embedded sensors and machine learning, enabling accurate shape estimation at 50Hz.

Contribution

It introduces a novel integration of low-cost sensors and neural networks for real-time 3D shape reconstruction of soft robots, addressing the challenge of infinite degrees-of-freedom.

Findings

Accurate 3D shape reconstruction from sensor signals.

Real-time processing at 50Hz on consumer devices.

Effective deformation parameterization for soft robots.

Abstract

Real-time proprioception is a challenging problem for soft robots, which have almost infinite degrees-of-freedom in body deformation. When multiple actuators are used, it becomes more difficult as deformation can also occur on actuators caused by interaction between each other. To tackle this problem, we present a method in this paper to sense and reconstruct 3D deformation on pneumatic soft robots by first integrating multiple low-cost sensors inside the chambers of pneumatic actuators and then using machine learning to convert the captured signals into shape parameters of soft robots. An exterior motion capture system is employed to generate the datasets for both training and testing. With the help of good shape parameterization, the 3D shape of a soft robot can be accurately reconstructed from signals obtained from multiple sensors. We demonstrate the effectiveness of this approach on two designs of soft robots -- a robotic joint and a deformable membrane. After parameterizing the deformation of these soft robots into compact shape parameters, we can effectively train the neural networks to reconstruct the 3D deformation from the sensor signals. The sensing and shape prediction pipeline can run at 50Hz in real-time on a consumer-level device.