Towards AI-controlled FES-restoration of arm movements: neuromechanics-based reinforcement learning for 3-D reaching

TL;DR

This paper introduces neuromechanical models and reinforcement learning to improve FES control for restoring arm movements, aiming for customizable, efficient, and generalizable solutions for different subjects and tasks.

Contribution

It presents customizable neuromechanical models and applies reinforcement learning as a novel, general control method for FES-induced arm movements in 3D environments.

Findings

Models capture significant FES dynamics with minimal computational cost

RL-based control enables customizable FES control across subjects

Demonstrated effectiveness in planar and 3D reaching tasks

Abstract

Reaching disabilities affect the quality of life. Functional Electrical Stimulation (FES) can restore lost motor functions. Yet, there remain challenges in controlling FES to induce desired movements. Neuromechanical models are valuable tools for developing FES control methods. However, focusing on the upper extremity areas, several existing models are either overly simplified or too computationally demanding for control purposes. Besides the model-related issues, finding a general method for governing the control rules for different tasks and subjects remains an engineering challenge. Here, we present our approach toward FES-based restoration of arm movements to address those fundamental issues in controlling FES. Firstly, we present our surface-FES-oriented neuromechanical models of human arms built using well-accepted, open-source software. The models are designed to capture significant dynamics in FES controls with minimal computational cost. Our models are customisable and can be used for testing different control methods. Secondly, we present the application of reinforcement learning (RL) as a general method for governing the control rules. In combination, our customisable models and RL-based control method open the possibility of delivering customised FES controls for different subjects and settings with minimal engineering intervention. We demonstrate our approach in planar and 3D settings.