A Robust and Scalable Attention Guided Deep Learning Framework for Movement Quality Assessment

TL;DR

This paper introduces a deep learning framework with novel data augmentation and a transformer-based architecture to improve automated movement quality assessment from skeletal data, aiding home rehabilitation.

Contribution

It presents three new skeletal data augmentation schemes and a transformer-based model with attention mechanisms for enhanced movement quality scoring.

Findings

12% improvement on UI-PRMD dataset

21% improvement on KIMORE dataset

Effective attention mechanism for key body parts

Abstract

Physical rehabilitation programs frequently begin with a brief stay in the hospital and continue with home-based rehabilitation. Lack of feedback on exercise correctness is a significant issue in home-based rehabilitation. Automated movement quality assessment (MQA) using skeletal movement data (hereafter referred to as skeletal data) collected via depth imaging devices can assist with home-based rehabilitation by providing the necessary quantitative feedback. This paper aims to use recent advances in deep learning to address the problem of MQA. Movement quality score generation is an essential component of MQA. We propose three novel skeletal data augmentation schemes. We show that using the proposed augmentations for generating movement quality scores result in significant performance boosts over existing methods. Finally, we propose a novel transformer based architecture for MQA. Four novel feature extractors are proposed and studied that allow the transformer network to operate on skeletal data. We show that adding the attention mechanism in the design of the proposed feature extractor allows the transformer network to pay attention to specific body parts that make a significant contribution towards executing a movement. We report an improvement in movement quality score prediction of 12% on UI-PRMD dataset and 21% on KIMORE dataset compared to the existing methods.