SENSORIMOTOR GRAPH: Action-Conditioned Graph Neural Network for Learning Robotic Soft Hand Dynamics

TL;DR

This paper introduces the Sensorimotor Graph, a graph neural network model that learns the structure of a soft robotic hand to improve dynamics prediction and robustness over traditional methods.

Contribution

It presents a novel graph neural network approach that leverages the system's structure for better modeling of soft robotic dynamics, incorporating order invariance and compositionality.

Findings

Outperforms non-structured baselines in dynamics prediction

More robust to configuration changes and node failures

Effective in modeling non-rigid soft robotic systems

Abstract

Soft robotics is a thriving branch of robotics which takes inspiration from nature and uses affordable flexible materials to design adaptable non-rigid robots. However, their flexible behavior makes these robots hard to model, which is essential for a precise actuation and for optimal control. For system modelling, learning-based approaches have demonstrated good results, yet they fail to consider the physical structure underlying the system as an inductive prior. In this work, we take inspiration from sensorimotor learning, and apply a Graph Neural Network to the problem of modelling a non-rigid kinematic chain (i.e. a robotic soft hand) taking advantage of two key properties: 1) the system is compositional, that is, it is composed of simple interacting parts connected by edges, 2) it is order invariant, i.e. only the structure of the system is relevant for predicting future trajectories. We denote our model as the 'Sensorimotor Graph' since it learns the system connectivity from observation and uses it for dynamics prediction. We validate our model in different scenarios and show that it outperforms the non-structured baselines in dynamics prediction while being more robust to configurational variations, tracking errors or node failures.