Biomechanic Posture Stabilisation via Iterative Training of Multi-policy Deep Reinforcement Learning Agents

TL;DR

This paper presents an iterative training method for deep reinforcement learning agents to improve biomechanical posture stabilization, significantly increasing standing duration and robustness against noise in digital musculoskeletal models.

Contribution

Introduces a novel iterative training procedure for deep reinforcement learning that enhances stability and generalization in biomechanical posture control tasks.

Findings

Standing duration increased from 4 to 348 seconds.

Agent generalized to perception and actuation noise for 108 seconds.

Method improves stability and robustness of digital musculoskeletal avatars.

Abstract

It is not until we become senior citizens do we recognise how much we took maintaining a simple standing posture for granted. It is truly fascinating to observe the magnitude of control the human brain exercises, in real time, to activate and deactivate the lower body muscles and solve a multi-link 3D inverted pendulum problem in order to maintain a stable standing posture. This realisation is even more apparent when training an artificial intelligence (AI) agent to maintain a standing posture of a digital musculoskeletal avatar due to the error propagation problem. In this work we address the error propagation problem by introducing an iterative training procedure for deep reinforcement learning which allows the agent to learn a finite set of actions and how to coordinate between them in order to achieve a stable standing posture. The proposed training approach allowed the agent to increase standing duration from 4 seconds using the traditional training method to 348 seconds using the proposed method. The proposed training method allowed the agent to generalise and accommodate perception and actuation noise for almost 108 seconds.