RePose: A Real-Time 3D Human Pose Estimation and Biomechanical Analysis Framework for Rehabilitation

TL;DR

RePose is a real-time 3D human pose estimation framework designed for rehabilitation, providing immediate feedback and biomechanical analysis to improve patient recovery using multi-camera RGB video input.

Contribution

It introduces an end-to-end real-time pose estimation pipeline with a fast multi-person tracking method and biomechanical analysis integrated into Unity for rehabilitation.

Findings

Achieves less than 1ms tracking per frame.

Effectively reduces pose estimation errors.

Enables real-time biomechanical feedback during rehabilitation.

Abstract

We propose a real-time 3D human pose estimation and motion analysis method termed RePose for rehabilitation training. It is capable of real-time monitoring and evaluation of patients'motion during rehabilitation, providing immediate feedback and guidance to assist patients in executing rehabilitation exercises correctly. Firstly, we introduce a unified pipeline for end-to-end real-time human pose estimation and motion analysis using RGB video input from multiple cameras which can be applied to the field of rehabilitation training. The pipeline can help to monitor and correct patients'actions, thus aiding them in regaining muscle strength and motor functions. Secondly, we propose a fast tracking method for medical rehabilitation scenarios with multiple-person interference, which requires less than 1ms for tracking for a single frame. Additionally, we modify SmoothNet for real-time posture estimation, effectively reducing pose estimation errors and restoring the patient's true motion state, making it visually smoother. Finally, we use Unity platform for real-time monitoring and evaluation of patients' motion during rehabilitation, and to display the muscle stress conditions to assist patients with their rehabilitation training.