Characterization of Human Balance through a Reinforcement Learning-based Muscle Controller

TL;DR

This paper introduces a reinforcement learning-based musculoskeletal model to assess human balance by analyzing the center of mass, offering a more comprehensive and objective method than traditional COP-based assessments.

Contribution

It presents a novel RL-trained muscle controller for human balance, exploring recovery limits and effects of impairments, advancing objective balance assessment methods.

Findings

Reinforcement learning effectively trains muscle controllers for balance recovery.

Balance recovery regions align with, but are more limited than, linear model predictions.

Muscle weakness and neural delays reduce balance recovery capabilities.

Abstract

Balance assessment during physical rehabilitation often relies on rubric-oriented battery tests to score a patient's physical capabilities, leading to subjectivity. While some objective balance assessments exist, they are often limited to tracking the center of pressure (COP), which does not fully capture the whole-body postural stability. This study explores the use of the center of mass (COM) state space and presents a promising avenue for monitoring the balance capabilities in humans. We employ a musculoskeletal model integrated with a balance controller, trained through reinforcement learning (RL), to investigate balancing capabilities. The RL framework consists of two interconnected neural networks governing balance recovery and muscle coordination respectively, trained using Proximal Policy Optimization (PPO) with reference state initialization, early termination, and multiple training strategies. By exploring recovery from random initial COM states (position and velocity) space for a trained controller, we obtain the final BR enclosing successful balance recovery trajectories. Comparing the BRs with analytical postural stability limits from a linear inverted pendulum model, we observe a similar trend in successful COM states but more limited ranges in the recoverable areas. We further investigate the effect of muscle weakness and neural excitation delay on the BRs, revealing reduced balancing capability in different regions. Overall, our approach of learning muscular balance controllers presents a promising new method for establishing balance recovery limits and objectively assessing balance capability in bipedal systems, particularly in humans.