# Deep multimodal biomechanical analysis for lower back pain rehabilitation to improve patients stability

## TL;DR

This paper introduces 3D-PoseFormer, an AI system for remote lower back pain rehabilitation using RGB and depth cameras to analyze physiotherapy exercises without wearable sensors.

## Contribution

The novel contribution is a deep multimodal framework combining RGB and depth data for real-time, sensor-free biomechanical analysis of LBP rehabilitation exercises.

## Key findings

- 3D-PoseFormer achieved 94.73% accuracy on the KIMORE dataset for LBP rehabilitation exercises.
- The system demonstrated 94.2% accuracy on the UTKinect-Action3D dataset, showing strong generalizability.
- It enables autonomous, continuous home-based monitoring without wearable sensors.

## Abstract

Advancements in artificial intelligence are transforming rehabilitation by enabling scalable, patient-centric solutions within modern healthcare systems. This study introduces 3D-PoseFormer, a deep multimodal framework designed for the telerehabilitation of individuals with lower back pain (LBP).

The proposed system performs automated data acquisition using synchronized RGB and depth video streams to enable real-time, markerless, and sensor-free analysis of physiotherapy exercises. From the depth sensing module, 3D body joint positions are extracted and used to generate SMPL-based mesh vertices for detailed biomechanical analysis and postural representation. Simultaneously, RGB frames are processed using keypoint detection algorithms—Shi-Tomasi, AKAZE, BRISK, SIFT, and Harris corner detection. Extracted features are enhanced through semantic contour analysis of segmented body parts to capture localized appearance-based information relevant to LBP therapy. The fused multimodal features are then passed to a Transformer-based machine learning model that captures temporal motion patterns for accurate exercise classification and human intention recognition.

The system removes the need for wearable sensors and supports autonomous, continuous monitoring in home-based rehabilitation. Validation on the KIMORE dataset (baseline, including rehabilitation exercises by patients with lower back pain), mRI dataset (rehabilitation exercises), and UTKinect-Action3D dataset (comprising diverse subjects and activity scenarios) achieved state-of-the-art accuracies of 94.73%, 91%, and 94.2%, respectively.

Results demonstrate the robustness, generalizability, and clinical potential of 3D-PoseFormer in AI-assisted rehabilitation, offering a scalable and intelligent healthcare system for remote physiotherapy and patient monitoring.