Toward smart composites: small-scale, untethered prediction and control for soft sensor/actuator systems

TL;DR

This paper introduces an open-source framework for in-material model predictive control of soft sensor/actuator systems using learned models and on-device computation, enabling untethered, high-bandwidth control with low memory footprint.

Contribution

It presents a novel approach combining neural network-based forward kinematics, an open-source compiler, and real-time control on microcontrollers for soft robotic systems.

Findings

Achieved high-bandwidth path tracking (≥120Hz) in experimental rigs.

Maintained small memory footprint (≤6.4% of flash memory).

Path following error within 2mm in tendon-based platform.

Abstract

We present formulation and open-source tools to achieve in-material model predictive control of sensor/actuator systems using learned forward kinematics and on-device computation. Microcontroller units (MCUs) that compute the prediction and control task while colocated with the sensors and actuators enable in-material untethered behaviors. In this approach, small parameter size neural network models learn forward kinematics offline. Our open-source compiler, nn4mc, generates code to offload these predictions onto MCUs. A Newton-Raphson solver then computes the control input in real time. We first benchmark this nonlinear control approach against a PID controller on a mass-spring-damper simulation. We then study experimental results on two experimental rigs with different sensing, actuation and computational hardware: a tendon-based platform with embedded LightLace sensors and a HASEL-based platform with magnetic sensors. Experimental results indicate effective high-bandwidth tracking of reference paths (greater than or equal to 120 Hz) with a small memory footprint (less than or equal to 6.4% of flash memory). The measured path following error does not exceed 2mm in the tendon-based platform. The simulated path following error does not exceed 1mm in the HASEL-based platform. The mean power consumption of this approach in an ARM Cortex-M4f device is 45.4 mW. This control approach is also compatible with Tensorflow Lite models and equivalent on-device code. In-material intelligence enables a new class of composites that infuse autonomy into structures and systems with refined artificial proprioception.