Towards AI-controlled FES-restoration of movements: Learning cycling stimulation pattern with reinforcement learning

TL;DR

This paper introduces an AI-driven approach to optimize FES cycling patterns using reinforcement learning and musculoskeletal models, eliminating manual tuning and improving cycling performance in simulations and experiments.

Contribution

It presents a novel two-phase method combining model-based reinforcement learning and real data fine-tuning for personalized FES cycling pattern optimization.

Findings

Model-based patterns are robust across configurations.

Experimental patterns outperform EMG-based patterns in speed.

Fine-tuning with 100 seconds of data improves cycling performance.

Abstract

Functional electrical stimulation (FES) has been increasingly integrated with other rehabilitation devices, including robots. FES cycling is one of the common FES applications in rehabilitation, which is performed by stimulating leg muscles in a certain pattern. The appropriate pattern varies across individuals and requires manual tuning which can be time-consuming and challenging for the individual user. Here, we present an AI-based method for finding the patterns, which requires no extra hardware or sensors. Our method has two phases, starting with finding model-based patterns using reinforcement learning and detailed musculoskeletal models. The models, built using open-source software, can be customised through our automated script and can be therefore used by non-technical individuals without extra cost. Next, our method fine-tunes the pattern using real cycling data. We test our both in simulation and experimentally on a stationary tricycle. In the simulation test, our method can robustly deliver model-based patterns for different cycling configurations. The experimental evaluation shows that our method can find a model-based pattern that induces higher cycling speed than an EMG-based pattern. By using just 100 seconds of cycling data, our method can deliver a fine-tuned pattern that gives better cycling performance. Beyond FES cycling, this work is a showcase, displaying the feasibility and potential of human-in-the-loop AI in real-world rehabilitation.