Reinforcement-Learning-Based Assistance Reduces Squat Effort with a Modular Hip--Knee Exoskeleton

TL;DR

This study demonstrates that a reinforcement learning-trained neural network controller for a modular Hip-Knee exoskeleton can effectively reduce physiological effort during squatting by approximately 10%, adapting assistance to individual kinematics.

Contribution

The paper introduces a novel RL-based neural network controller for a modular exoskeleton that personalizes assistance during squatting tasks, showing promising reductions in effort.

Findings

RL controller adapts assistance torque to individual kinematics

Assisted squatting reduces metabolic rate by ~10%

Assistance leads to decreased squat depth

Abstract

Squatting is one of the most demanding lower-limb movements, requiring substantial muscular effort and coordination. Reducing the physical demands of this task through intelligent and personalized assistance has significant implications, particularly in industries involving repetitive low-level assembly activities. In this study, we evaluated the effectiveness of a neural network controller for a modular Hip-Knee exoskeleton designed to assist squatting tasks. The neural network controller was trained via reinforcement learning (RL) in a physics-based, human-exoskeleton interaction simulation environment. The controller generated real-time hip and knee assistance torques based on recent joint-angle and velocity histories. Five healthy adults performed three-minute metronome-guided squats under three conditions: (1) no exoskeleton (No-Exo), (2) exoskeleton with Zero-Torque, and (3) exoskeleton with active assistance (Assistance). Physiological effort was assessed using indirect calorimetry and heart rate monitoring, alongside concurrent kinematic data collection. Results show that the RL-based controller adapts to individuals by producing torque profiles tailored to each subject's kinematics and timing. Compared with the Zero-Torque and No-Exo condition, active assistance reduced the net metabolic rate by approximately 10%, with minor reductions observed in heart rate. However, assisted trials also exhibited reduced squat depth, reflected by smaller hip and knee flexion. These preliminary findings suggest that the proposed controller can effectively lower physiological effort during repetitive squatting, motivating further improvements in both hardware design and control strategies.