Learning User Interaction Forces using Vision for a Soft Finger Exosuit

TL;DR

This paper presents a vision-based, learning-driven approach to estimate contact forces in a soft finger exosuit, enabling non-intrusive, real-time force sensing for improved control and interaction modeling.

Contribution

It introduces a novel image-based framework trained on simulated data to accurately estimate distributed contact forces in a soft exosuit, generalizing across shapes and noise conditions.

Findings

Accurately estimated interaction forces from low-res images.

Generalized to unseen shapes and actuation levels.

Enabled closed-loop control using vision-based force estimation.

Abstract

Wearable assistive devices are increasingly becoming softer. Modelling their interface with human tissue is necessary to capture transmission of dynamic assistance. However, their nonlinear and compliant nature makes both physical modeling and embedded sensing challenging. In this paper, we develop a image-based, learning-based framework to estimate distributed contact forces for a finger-exosuit system. We used the SoRoSim toolbox to generate a diverse dataset of exosuit geometries and actuation scenarios for training. The method accurately estimated interaction forces across multiple contact locations from low-resolution grayscale images, was able to generalize to unseen shapes and actuation levels, and remained robust under visual noise and contrast variations. We integrated the model into a feedback controller, and found that the vision-based estimator functions as a surrogate force sensor for closed-loop control. This approach could be used as a non-intrusive alternative for real-time force estimation for exosuits.