Automated Control of Rehabilitation Process in Physical Therapy Using a Novel Human Skeleton-Based Balanced Time Warping Algorithm

TL;DR

This paper introduces a new computer vision system that uses a skeleton-based algorithm to monitor and evaluate physical therapy exercises in real time, improving rehabilitation accuracy and efficiency.

Contribution

A novel Human Skeleton-based Balanced Time Warping algorithm for automated rehabilitation monitoring without pre-alignment or calibration.

Findings

The system achieved a high Spearman’s rank correlation coefficient of 0.977 between computed dissimilarity and exercise accuracy.

The method effectively clusters exercise performance into 'good,' 'intermediate,' and 'bad' accuracy levels.

The approach enables real-time feedback and reduces therapist workload while supporting remote rehabilitation monitoring.

Abstract

Physical therapy is a critical component of medical rehabilitation, aiding recovery from conditions such as stroke, spinal cord injuries, and musculoskeletal disorders. Effective rehabilitation requires precise monitoring of patient performance to ensure exercises are executed correctly and progress is accurately assessed. This paper presents a novel automated system for controlling the rehabilitation process and evaluating physical therapy exercise quality using computer vision and a customized Human Skeleton-based Balanced Time Warping algorithm. The proposed method quantitatively assesses the similarity between a physiotherapist and patient performance by analyzing skeletal motion data extracted from RGB-D video sequences without requiring pre-alignment or sensor-specific calibration. A motion-dependent, weighted Euclidean distance between 3D skeletal models is used to compute pose dissimilarity, while a modified DTW approach aligns temporal sequences and evaluates dynamic consistency. The total dissimilarity measure is a balanced combination of posture (DP) and dynamics (DT) components. Evaluated on a custom dataset of 136 video recordings from 23 participants performing exercises in sitting and standing positions under varying performance accuracy levels (“good,” “intermediate,” and “bad”), the system demonstrates the strong clustering of accuracy levels. Proposed dissimilarity, together with a fixed reference element (physiotherapist), induces a natural non-strict order on the set of distances between patients and physiotherapists. A high value of Spearman’s rank correlation coefficient between computed dissimilarity and execution accuracy (0.977) indicates that this method is suitable for assessing exercise performance accuracy and for adequately evaluating the patient’s rehabilitation progress. The method enables objective, real-time feedback, reduces therapist workload, and supports remote monitoring, offering a scalable solution for personalized rehabilitation. Future work will involve clinical validation with post-stroke and cardiac patients.

Figures