Towards Expedited Impedance Tuning of a Robotic Prosthesis for Personalized Gait Assistance by Reinforcement Learning Control

TL;DR

This paper introduces PICE, a reinforcement learning method that significantly speeds up the process of tuning control parameters for robotic knee prostheses, enhancing personalization and safety in clinical settings.

Contribution

The paper presents PICE, a novel policy iteration method with embedded constraints, improving tuning efficiency for prosthetic control through online and offline implementations.

Findings

PICE reduced tuning time significantly in human tests.

PICE demonstrated robustness across different tasks and users.

The method ensures safety by maintaining positive semidefiniteness during policy evaluation.

Abstract

Personalizing medical devices such as lower limb wearable robots is challenging. While the initial feasibility of automating the process of knee prosthesis control parameter tuning has been demonstrated in a principled way, the next critical issue is to improve tuning efficiency and speed it up for the human user, in clinic settings, while maintaining human safety. We, therefore, propose a policy iteration with constraint embedded (PICE) method as an innovative solution to the problem under the framework of reinforcement learning. Central to PICE is the use of a projected Bellman equation with a constraint of assuring positive semidefiniteness of performance values during policy evaluation. Additionally, we developed both online and offline PICE implementations that provide additional flexibility for the designer to fully utilize measurement data, either from on-policy or off-policy, to further improve PICE tuning efficiency. Our human subject testing showed that the PICE provided effective policies with significantly reduced tuning time. For the first time, we also experimentally evaluated and demonstrated the robustness of the deployed policies by applying them to different tasks and users. Putting it together, our new way of problem solving has been effective as PICE has demonstrated its potential toward truly automating the process of control parameter tuning for robotic knee prosthesis users.